2023-02-09 09:53:30,374 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-02-09 09:53:30,658 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390 2023-02-09 09:53:30,672 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-09 09:53:30,672 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/cluster_355d5ed8-3bf0-2bc4-8778-241b2c013167, deleteOnExit=true 2023-02-09 09:53:30,673 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-09 09:53:30,673 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/test.cache.data in system properties and HBase conf 2023-02-09 09:53:30,674 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/hadoop.tmp.dir in system properties and HBase conf 2023-02-09 09:53:30,674 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/hadoop.log.dir in system properties and HBase conf 2023-02-09 09:53:30,674 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-09 09:53:30,675 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-09 09:53:30,675 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-09 09:53:30,784 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-09 09:53:31,206 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-09 09:53:31,212 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-09 09:53:31,212 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-09 09:53:31,213 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-09 09:53:31,213 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 09:53:31,214 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-09 09:53:31,214 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-09 09:53:31,214 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 09:53:31,215 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 09:53:31,215 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-09 09:53:31,216 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/nfs.dump.dir in system properties and HBase conf 2023-02-09 09:53:31,216 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/java.io.tmpdir in system properties and HBase conf 2023-02-09 09:53:31,217 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 09:53:31,217 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-09 09:53:31,217 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-09 09:53:31,781 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 09:53:31,787 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 09:53:32,071 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-09 09:53:32,251 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-09 09:53:32,266 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 09:53:32,320 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 09:53:32,366 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/java.io.tmpdir/Jetty_localhost_45277_hdfs____.n3xo11/webapp 2023-02-09 09:53:32,524 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45277 2023-02-09 09:53:32,561 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 09:53:32,562 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 09:53:33,022 WARN [Listener at localhost/37539] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 09:53:33,131 WARN [Listener at localhost/37539] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 09:53:33,149 WARN [Listener at localhost/37539] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 09:53:33,156 INFO [Listener at localhost/37539] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 09:53:33,161 INFO [Listener at localhost/37539] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/java.io.tmpdir/Jetty_localhost_33979_datanode____gl6w2q/webapp 2023-02-09 09:53:33,256 INFO [Listener at localhost/37539] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33979 2023-02-09 09:53:33,567 WARN [Listener at localhost/42557] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 09:53:33,990 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdf7e5e91d18d5e5d: Processing first storage report for DS-e72941d8-0a81-4c47-ba2e-0c151a1d08a8 from datanode 41a2457e-4972-4171-9111-56b0322eb7a3 2023-02-09 09:53:33,992 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdf7e5e91d18d5e5d: from storage DS-e72941d8-0a81-4c47-ba2e-0c151a1d08a8 node DatanodeRegistration(127.0.0.1:44643, datanodeUuid=41a2457e-4972-4171-9111-56b0322eb7a3, infoPort=39253, infoSecurePort=0, ipcPort=42557, storageInfo=lv=-57;cid=testClusterID;nsid=1792492071;c=1675936411864), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-09 09:53:33,992 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdf7e5e91d18d5e5d: Processing first storage report for DS-35000e22-2fe1-4913-a379-e7b39ab985d8 from datanode 41a2457e-4972-4171-9111-56b0322eb7a3 2023-02-09 09:53:33,992 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdf7e5e91d18d5e5d: from storage DS-35000e22-2fe1-4913-a379-e7b39ab985d8 node DatanodeRegistration(127.0.0.1:44643, datanodeUuid=41a2457e-4972-4171-9111-56b0322eb7a3, infoPort=39253, infoSecurePort=0, ipcPort=42557, storageInfo=lv=-57;cid=testClusterID;nsid=1792492071;c=1675936411864), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 09:53:34,047 DEBUG [Listener at localhost/42557] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390 2023-02-09 09:53:34,108 INFO [Listener at localhost/42557] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/cluster_355d5ed8-3bf0-2bc4-8778-241b2c013167/zookeeper_0, clientPort=64170, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/cluster_355d5ed8-3bf0-2bc4-8778-241b2c013167/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/cluster_355d5ed8-3bf0-2bc4-8778-241b2c013167/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-09 09:53:34,121 INFO [Listener at localhost/42557] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=64170 2023-02-09 09:53:34,131 INFO [Listener at localhost/42557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 09:53:34,134 INFO [Listener at localhost/42557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 09:53:34,785 INFO [Listener at localhost/42557] util.FSUtils(479): Created version file at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b with version=8 2023-02-09 09:53:34,786 INFO [Listener at localhost/42557] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/hbase-staging 2023-02-09 09:53:35,082 INFO [Listener at localhost/42557] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-09 09:53:35,538 INFO [Listener at localhost/42557] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-02-09 09:53:35,569 INFO [Listener at localhost/42557] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 09:53:35,570 INFO [Listener at localhost/42557] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 09:53:35,570 INFO [Listener at localhost/42557] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 09:53:35,570 INFO [Listener at localhost/42557] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 09:53:35,570 INFO [Listener at localhost/42557] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 09:53:35,708 INFO [Listener at localhost/42557] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 09:53:35,780 DEBUG [Listener at localhost/42557] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-09 09:53:35,872 INFO [Listener at localhost/42557] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33303 2023-02-09 09:53:35,882 INFO [Listener at localhost/42557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 09:53:35,884 INFO [Listener at localhost/42557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 09:53:35,904 INFO [Listener at localhost/42557] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33303 connecting to ZooKeeper ensemble=127.0.0.1:64170 2023-02-09 09:53:35,943 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:333030x0, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 09:53:35,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:33303-0x100388023520000 connected 2023-02-09 09:53:35,971 DEBUG [Listener at localhost/42557] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 09:53:35,972 DEBUG [Listener at localhost/42557] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 09:53:35,975 DEBUG [Listener at localhost/42557] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 09:53:35,983 DEBUG [Listener at localhost/42557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33303 2023-02-09 09:53:35,983 DEBUG [Listener at localhost/42557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33303 2023-02-09 09:53:35,983 DEBUG [Listener at localhost/42557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33303 2023-02-09 09:53:35,984 DEBUG [Listener at localhost/42557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33303 2023-02-09 09:53:35,984 DEBUG [Listener at localhost/42557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33303 2023-02-09 09:53:35,991 INFO [Listener at localhost/42557] master.HMaster(439): hbase.rootdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b, hbase.cluster.distributed=false 2023-02-09 09:53:36,062 INFO [Listener at localhost/42557] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-02-09 09:53:36,062 INFO [Listener at localhost/42557] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 09:53:36,062 INFO [Listener at localhost/42557] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 09:53:36,063 INFO [Listener at localhost/42557] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 09:53:36,063 INFO [Listener at localhost/42557] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 09:53:36,063 INFO [Listener at localhost/42557] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 09:53:36,067 INFO [Listener at localhost/42557] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 09:53:36,070 INFO [Listener at localhost/42557] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42719 2023-02-09 09:53:36,073 INFO [Listener at localhost/42557] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 09:53:36,079 DEBUG [Listener at localhost/42557] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 09:53:36,080 INFO [Listener at localhost/42557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 09:53:36,083 INFO [Listener at localhost/42557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 09:53:36,084 INFO [Listener at localhost/42557] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42719 connecting to ZooKeeper ensemble=127.0.0.1:64170 2023-02-09 09:53:36,088 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:427190x0, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 09:53:36,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:42719-0x100388023520001 connected 2023-02-09 09:53:36,089 DEBUG [Listener at localhost/42557] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 09:53:36,090 DEBUG [Listener at localhost/42557] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 09:53:36,090 DEBUG [Listener at localhost/42557] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 09:53:36,091 DEBUG [Listener at localhost/42557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42719 2023-02-09 09:53:36,091 DEBUG [Listener at localhost/42557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42719 2023-02-09 09:53:36,092 DEBUG [Listener at localhost/42557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42719 2023-02-09 09:53:36,092 DEBUG [Listener at localhost/42557] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42719 2023-02-09 09:53:36,092 DEBUG [Listener at localhost/42557] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42719 2023-02-09 09:53:36,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33303,1675936414925 2023-02-09 09:53:36,103 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 09:53:36,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33303,1675936414925 2023-02-09 09:53:36,124 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 09:53:36,124 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 09:53:36,124 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 09:53:36,125 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 09:53:36,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33303,1675936414925 from backup master directory 2023-02-09 09:53:36,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 09:53:36,129 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33303,1675936414925 2023-02-09 09:53:36,129 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 09:53:36,130 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 09:53:36,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,33303,1675936414925 2023-02-09 09:53:36,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-09 09:53:36,134 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-09 09:53:36,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/hbase.id with ID: bf71699d-f34a-4a4d-a38f-3deed4cb2204 2023-02-09 09:53:36,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 09:53:36,270 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 09:53:36,709 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x594f98ad to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:53:36,742 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@71988fe8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:53:36,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 09:53:36,766 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-09 09:53:36,782 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-09 09:53:36,782 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-09 09:53:36,784 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 09:53:36,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 09:53:36,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 09:53:36,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/data/master/store-tmp 2023-02-09 09:53:36,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:36,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 09:53:36,846 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 09:53:36,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 09:53:36,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 09:53:36,847 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 09:53:36,847 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 09:53:36,847 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675936416846Disabling compacts and flushes for region at 1675936416846Disabling writes for close at 1675936416847 (+1 ms)Writing region close event to WAL at 1675936416847Closed at 1675936416847 2023-02-09 09:53:36,848 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/WALs/jenkins-hbase4.apache.org,33303,1675936414925 2023-02-09 09:53:36,868 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33303%2C1675936414925, suffix=, logDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/WALs/jenkins-hbase4.apache.org,33303,1675936414925, archiveDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/oldWALs, maxLogs=10 2023-02-09 09:53:36,918 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44643,DS-e72941d8-0a81-4c47-ba2e-0c151a1d08a8,DISK] 2023-02-09 09:53:36,925 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-09 09:53:36,982 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/WALs/jenkins-hbase4.apache.org,33303,1675936414925/jenkins-hbase4.apache.org%2C33303%2C1675936414925.1675936416877 2023-02-09 09:53:36,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44643,DS-e72941d8-0a81-4c47-ba2e-0c151a1d08a8,DISK]] 2023-02-09 09:53:36,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:53:36,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:36,987 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 09:53:36,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 09:53:37,044 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-09 09:53:37,051 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-09 09:53:37,074 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-09 09:53:37,087 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:37,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 09:53:37,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 09:53:37,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 09:53:37,114 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:53:37,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65434534, jitterRate=-0.024949461221694946}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:53:37,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675936417014Initializing all the Stores at 1675936417017 (+3 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675936417018 (+1 ms)Cleaning up temporary data from old regions at 1675936417102 (+84 ms)Cleaning up detritus from prior splits at 1675936417102Region opened successfully at 1675936417115 (+13 ms) 2023-02-09 09:53:37,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-09 09:53:37,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-09 09:53:37,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-09 09:53:37,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 09:53:37,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-09 09:53:37,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-02-09 09:53:37,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-09 09:53:37,195 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-09 09:53:37,200 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-09 09:53:37,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-09 09:53:37,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-09 09:53:37,228 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-09 09:53:37,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-09 09:53:37,237 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-09 09:53:37,241 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 09:53:37,242 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-09 09:53:37,242 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-09 09:53:37,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-09 09:53:37,257 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 09:53:37,257 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 09:53:37,258 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 09:53:37,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,33303,1675936414925, sessionid=0x100388023520000, setting cluster-up flag (Was=false) 2023-02-09 09:53:37,272 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 09:53:37,278 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-09 09:53:37,279 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33303,1675936414925 2023-02-09 09:53:37,285 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 09:53:37,289 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-09 09:53:37,290 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33303,1675936414925 2023-02-09 09:53:37,293 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/.tmp 2023-02-09 09:53:37,297 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(952): ClusterId : bf71699d-f34a-4a4d-a38f-3deed4cb2204 2023-02-09 09:53:37,302 DEBUG [RS:0;jenkins-hbase4:42719] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 09:53:37,308 DEBUG [RS:0;jenkins-hbase4:42719] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 09:53:37,308 DEBUG [RS:0;jenkins-hbase4:42719] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 09:53:37,311 DEBUG [RS:0;jenkins-hbase4:42719] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 09:53:37,312 DEBUG [RS:0;jenkins-hbase4:42719] zookeeper.ReadOnlyZKClient(139): Connect 0x4b22ac13 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:53:37,316 DEBUG [RS:0;jenkins-hbase4:42719] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@45e54b07, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:53:37,317 DEBUG [RS:0;jenkins-hbase4:42719] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@41dac763, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 09:53:37,340 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42719 2023-02-09 09:53:37,343 INFO [RS:0;jenkins-hbase4:42719] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 09:53:37,343 INFO [RS:0;jenkins-hbase4:42719] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 09:53:37,343 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 09:53:37,346 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33303,1675936414925 with isa=jenkins-hbase4.apache.org/172.31.14.131:42719, startcode=1675936416061 2023-02-09 09:53:37,361 DEBUG [RS:0;jenkins-hbase4:42719] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 09:53:37,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-09 09:53:37,413 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58433, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 09:53:37,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 09:53:37,423 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 09:53:37,423 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 09:53:37,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 09:53:37,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 09:53:37,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-09 09:53:37,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 09:53:37,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675936447429 2023-02-09 09:53:37,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-09 09:53:37,434 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 09:53:37,435 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-09 09:53:37,440 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 09:53:37,442 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-09 09:53:37,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-09 09:53:37,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-09 09:53:37,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-09 09:53:37,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-09 09:53:37,450 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:37,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-09 09:53:37,452 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 09:53:37,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-09 09:53:37,453 WARN [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 09:53:37,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-09 09:53:37,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-09 09:53:37,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-09 09:53:37,457 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675936417457,5,FailOnTimeoutGroup] 2023-02-09 09:53:37,457 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675936417457,5,FailOnTimeoutGroup] 2023-02-09 09:53:37,457 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:37,457 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-09 09:53:37,458 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:37,458 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:37,554 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33303,1675936414925 with isa=jenkins-hbase4.apache.org/172.31.14.131:42719, startcode=1675936416061 2023-02-09 09:53:37,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:37,565 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b 2023-02-09 09:53:37,565 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:37539 2023-02-09 09:53:37,565 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 09:53:37,570 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 09:53:37,570 DEBUG [RS:0;jenkins-hbase4:42719] zookeeper.ZKUtil(162): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:37,571 WARN [RS:0;jenkins-hbase4:42719] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 09:53:37,571 INFO [RS:0;jenkins-hbase4:42719] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 09:53:37,571 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1947): logDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/WALs/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:37,572 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42719,1675936416061] 2023-02-09 09:53:37,580 DEBUG [RS:0;jenkins-hbase4:42719] zookeeper.ZKUtil(162): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:37,589 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 09:53:37,597 INFO [RS:0;jenkins-hbase4:42719] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 09:53:37,616 INFO [RS:0;jenkins-hbase4:42719] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 09:53:37,619 INFO [RS:0;jenkins-hbase4:42719] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:53:37,619 INFO [RS:0;jenkins-hbase4:42719] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:37,620 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 09:53:37,626 INFO [RS:0;jenkins-hbase4:42719] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-02-09 09:53:37,627 DEBUG [RS:0;jenkins-hbase4:42719] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,627 DEBUG [RS:0;jenkins-hbase4:42719] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,627 DEBUG [RS:0;jenkins-hbase4:42719] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,627 DEBUG [RS:0;jenkins-hbase4:42719] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,627 DEBUG [RS:0;jenkins-hbase4:42719] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,627 DEBUG [RS:0;jenkins-hbase4:42719] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 09:53:37,627 DEBUG [RS:0;jenkins-hbase4:42719] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,628 DEBUG [RS:0;jenkins-hbase4:42719] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,628 DEBUG [RS:0;jenkins-hbase4:42719] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,628 DEBUG [RS:0;jenkins-hbase4:42719] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 09:53:37,629 INFO [RS:0;jenkins-hbase4:42719] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:37,629 INFO [RS:0;jenkins-hbase4:42719] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:37,629 INFO [RS:0;jenkins-hbase4:42719] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:37,645 INFO [RS:0;jenkins-hbase4:42719] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 09:53:37,648 INFO [RS:0;jenkins-hbase4:42719] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42719,1675936416061-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:37,664 INFO [RS:0;jenkins-hbase4:42719] regionserver.Replication(203): jenkins-hbase4.apache.org,42719,1675936416061 started 2023-02-09 09:53:37,664 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,42719,1675936416061, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42719, sessionid=0x100388023520001 2023-02-09 09:53:37,664 DEBUG [RS:0;jenkins-hbase4:42719] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 09:53:37,664 DEBUG [RS:0;jenkins-hbase4:42719] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:37,664 DEBUG [RS:0;jenkins-hbase4:42719] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42719,1675936416061' 2023-02-09 09:53:37,664 DEBUG [RS:0;jenkins-hbase4:42719] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:37,665 DEBUG [RS:0;jenkins-hbase4:42719] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:37,665 DEBUG [RS:0;jenkins-hbase4:42719] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 09:53:37,665 DEBUG [RS:0;jenkins-hbase4:42719] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 09:53:37,665 DEBUG [RS:0;jenkins-hbase4:42719] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:37,665 DEBUG [RS:0;jenkins-hbase4:42719] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42719,1675936416061' 2023-02-09 09:53:37,665 DEBUG [RS:0;jenkins-hbase4:42719] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 09:53:37,666 DEBUG [RS:0;jenkins-hbase4:42719] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 09:53:37,666 DEBUG [RS:0;jenkins-hbase4:42719] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 09:53:37,666 INFO [RS:0;jenkins-hbase4:42719] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 09:53:37,666 INFO [RS:0;jenkins-hbase4:42719] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 09:53:37,777 INFO [RS:0;jenkins-hbase4:42719] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42719%2C1675936416061, suffix=, logDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/WALs/jenkins-hbase4.apache.org,42719,1675936416061, archiveDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/oldWALs, maxLogs=32 2023-02-09 09:53:37,794 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44643,DS-e72941d8-0a81-4c47-ba2e-0c151a1d08a8,DISK] 2023-02-09 09:53:37,798 INFO [RS:0;jenkins-hbase4:42719] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/WALs/jenkins-hbase4.apache.org,42719,1675936416061/jenkins-hbase4.apache.org%2C42719%2C1675936416061.1675936417779 2023-02-09 09:53:37,798 DEBUG [RS:0;jenkins-hbase4:42719] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44643,DS-e72941d8-0a81-4c47-ba2e-0c151a1d08a8,DISK]] 2023-02-09 09:53:37,871 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 09:53:37,872 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 09:53:37,873 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b 2023-02-09 09:53:38,285 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:38,288 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 09:53:38,291 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/info 2023-02-09 09:53:38,292 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 09:53:38,293 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:38,293 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 09:53:38,295 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/rep_barrier 2023-02-09 09:53:38,296 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 09:53:38,297 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:38,297 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 09:53:38,299 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/table 2023-02-09 09:53:38,299 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 09:53:38,300 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:38,302 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740 2023-02-09 09:53:38,303 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740 2023-02-09 09:53:38,306 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 09:53:38,308 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 09:53:38,312 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:53:38,313 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74025825, jitterRate=0.10307075083255768}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 09:53:38,313 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675936418286Initializing all the Stores at 1675936418287 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936418287Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675936418288 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936418288Cleaning up temporary data from old regions at 1675936418304 (+16 ms)Cleaning up detritus from prior splits at 1675936418305 (+1 ms)Region opened successfully at 1675936418313 (+8 ms) 2023-02-09 09:53:38,313 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 09:53:38,313 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 09:53:38,313 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 09:53:38,313 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 09:53:38,313 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 09:53:38,314 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 09:53:38,314 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675936418313Disabling compacts and flushes for region at 1675936418313Disabling writes for close at 1675936418313Writing region close event to WAL at 1675936418314 (+1 ms)Closed at 1675936418314 2023-02-09 09:53:38,319 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 09:53:38,319 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-09 09:53:38,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-09 09:53:38,340 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-09 09:53:38,342 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 09:53:38,494 DEBUG [jenkins-hbase4:33303] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-02-09 09:53:38,497 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42719,1675936416061, state=OPENING 2023-02-09 09:53:38,506 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-09 09:53:38,508 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 09:53:38,508 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 09:53:38,510 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:53:38,694 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:38,696 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 09:53:38,700 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41660, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 09:53:38,712 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-09 09:53:38,712 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 09:53:38,715 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42719%2C1675936416061.meta, suffix=.meta, logDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/WALs/jenkins-hbase4.apache.org,42719,1675936416061, archiveDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/oldWALs, maxLogs=32 2023-02-09 09:53:38,732 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44643,DS-e72941d8-0a81-4c47-ba2e-0c151a1d08a8,DISK] 2023-02-09 09:53:38,736 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/WALs/jenkins-hbase4.apache.org,42719,1675936416061/jenkins-hbase4.apache.org%2C42719%2C1675936416061.meta.1675936418716.meta 2023-02-09 09:53:38,737 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44643,DS-e72941d8-0a81-4c47-ba2e-0c151a1d08a8,DISK]] 2023-02-09 09:53:38,737 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:53:38,739 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-09 09:53:38,755 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-09 09:53:38,760 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-09 09:53:38,765 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-09 09:53:38,765 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:38,765 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-09 09:53:38,765 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-09 09:53:38,768 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 09:53:38,769 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/info 2023-02-09 09:53:38,769 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/info 2023-02-09 09:53:38,770 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 09:53:38,771 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:38,771 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 09:53:38,773 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/rep_barrier 2023-02-09 09:53:38,773 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/rep_barrier 2023-02-09 09:53:38,773 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 09:53:38,774 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:38,774 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 09:53:38,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/table 2023-02-09 09:53:38,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/table 2023-02-09 09:53:38,776 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 09:53:38,777 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:38,779 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740 2023-02-09 09:53:38,781 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740 2023-02-09 09:53:38,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 09:53:38,788 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 09:53:38,789 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69603420, jitterRate=0.037171781063079834}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 09:53:38,789 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675936418765Writing region info on filesystem at 1675936418766 (+1 ms)Initializing all the Stores at 1675936418767 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936418767Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675936418767Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936418767Cleaning up temporary data from old regions at 1675936418783 (+16 ms)Cleaning up detritus from prior splits at 1675936418783Running coprocessor post-open hooks at 1675936418789 (+6 ms)Region opened successfully at 1675936418789 2023-02-09 09:53:38,798 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675936418687 2023-02-09 09:53:38,816 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-09 09:53:38,816 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-09 09:53:38,817 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42719,1675936416061, state=OPEN 2023-02-09 09:53:38,819 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-09 09:53:38,820 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 09:53:38,823 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-09 09:53:38,823 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42719,1675936416061 in 309 msec 2023-02-09 09:53:38,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-09 09:53:38,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 497 msec 2023-02-09 09:53:38,833 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.4900 sec 2023-02-09 09:53:38,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675936416119, completionTime=-1 2023-02-09 09:53:38,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-02-09 09:53:38,834 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 09:53:38,903 DEBUG [hconnection-0x737c9c8-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:53:38,907 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41666, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:53:38,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-02-09 09:53:38,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675936478925 2023-02-09 09:53:38,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675936538925 2023-02-09 09:53:38,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 91 msec 2023-02-09 09:53:38,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33303,1675936414925-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:38,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33303,1675936414925-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:38,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33303,1675936414925-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:38,951 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33303, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:38,951 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-09 09:53:38,957 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-09 09:53:38,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-09 09:53:38,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 09:53:38,976 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-09 09:53:38,978 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:53:38,980 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:53:39,002 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf 2023-02-09 09:53:39,005 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf empty. 2023-02-09 09:53:39,006 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf 2023-02-09 09:53:39,006 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-09 09:53:39,428 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-09 09:53:39,430 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 927ca45314341fe03c1d443d120d31cf, NAME => 'hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:53:39,443 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:39,443 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 927ca45314341fe03c1d443d120d31cf, disabling compactions & flushes 2023-02-09 09:53:39,443 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:53:39,443 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:53:39,443 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. after waiting 0 ms 2023-02-09 09:53:39,443 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:53:39,443 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:53:39,443 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 927ca45314341fe03c1d443d120d31cf: Waiting for close lock at 1675936419443Disabling compacts and flushes for region at 1675936419443Disabling writes for close at 1675936419443Writing region close event to WAL at 1675936419443Closed at 1675936419443 2023-02-09 09:53:39,447 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:53:39,462 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675936419450"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936419450"}]},"ts":"1675936419450"} 2023-02-09 09:53:39,488 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 09:53:39,489 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:53:39,494 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936419490"}]},"ts":"1675936419490"} 2023-02-09 09:53:39,498 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-09 09:53:39,506 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=927ca45314341fe03c1d443d120d31cf, ASSIGN}] 2023-02-09 09:53:39,508 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=927ca45314341fe03c1d443d120d31cf, ASSIGN 2023-02-09 09:53:39,510 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=927ca45314341fe03c1d443d120d31cf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:53:39,661 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=927ca45314341fe03c1d443d120d31cf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:39,661 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675936419661"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936419661"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936419661"}]},"ts":"1675936419661"} 2023-02-09 09:53:39,667 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 927ca45314341fe03c1d443d120d31cf, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:53:39,826 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:53:39,827 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 927ca45314341fe03c1d443d120d31cf, NAME => 'hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:53:39,828 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 927ca45314341fe03c1d443d120d31cf 2023-02-09 09:53:39,828 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:39,828 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 927ca45314341fe03c1d443d120d31cf 2023-02-09 09:53:39,828 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 927ca45314341fe03c1d443d120d31cf 2023-02-09 09:53:39,830 INFO [StoreOpener-927ca45314341fe03c1d443d120d31cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 927ca45314341fe03c1d443d120d31cf 2023-02-09 09:53:39,832 DEBUG [StoreOpener-927ca45314341fe03c1d443d120d31cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf/info 2023-02-09 09:53:39,832 DEBUG [StoreOpener-927ca45314341fe03c1d443d120d31cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf/info 2023-02-09 09:53:39,833 INFO [StoreOpener-927ca45314341fe03c1d443d120d31cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 927ca45314341fe03c1d443d120d31cf columnFamilyName info 2023-02-09 09:53:39,833 INFO [StoreOpener-927ca45314341fe03c1d443d120d31cf-1] regionserver.HStore(310): Store=927ca45314341fe03c1d443d120d31cf/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:39,835 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf 2023-02-09 09:53:39,836 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf 2023-02-09 09:53:39,840 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 927ca45314341fe03c1d443d120d31cf 2023-02-09 09:53:39,843 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:53:39,844 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 927ca45314341fe03c1d443d120d31cf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74198623, jitterRate=0.10564564168453217}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:53:39,844 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 927ca45314341fe03c1d443d120d31cf: Running coprocessor pre-open hook at 1675936419828Writing region info on filesystem at 1675936419828Initializing all the Stores at 1675936419829 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675936419830 (+1 ms)Cleaning up temporary data from old regions at 1675936419836 (+6 ms)Cleaning up detritus from prior splits at 1675936419837 (+1 ms)Running coprocessor post-open hooks at 1675936419844 (+7 ms)Region opened successfully at 1675936419844 2023-02-09 09:53:39,846 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf., pid=6, masterSystemTime=1675936419820 2023-02-09 09:53:39,850 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:53:39,850 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:53:39,851 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=927ca45314341fe03c1d443d120d31cf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:39,852 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675936419851"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936419851"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936419851"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936419851"}]},"ts":"1675936419851"} 2023-02-09 09:53:39,859 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-09 09:53:39,859 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 927ca45314341fe03c1d443d120d31cf, server=jenkins-hbase4.apache.org,42719,1675936416061 in 188 msec 2023-02-09 09:53:39,862 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-09 09:53:39,862 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=927ca45314341fe03c1d443d120d31cf, ASSIGN in 353 msec 2023-02-09 09:53:39,863 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:53:39,864 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936419864"}]},"ts":"1675936419864"} 2023-02-09 09:53:39,866 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-09 09:53:39,871 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:53:39,873 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 904 msec 2023-02-09 09:53:39,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-09 09:53:39,879 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-09 09:53:39,879 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 09:53:39,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-09 09:53:39,931 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 09:53:39,937 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 31 msec 2023-02-09 09:53:39,946 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-09 09:53:39,957 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 09:53:39,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-02-09 09:53:39,971 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-09 09:53:39,974 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-09 09:53:39,975 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 3.845sec 2023-02-09 09:53:39,976 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-09 09:53:39,978 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-09 09:53:39,978 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-09 09:53:39,979 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33303,1675936414925-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-09 09:53:39,980 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33303,1675936414925-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-09 09:53:39,990 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-09 09:53:40,003 DEBUG [Listener at localhost/42557] zookeeper.ReadOnlyZKClient(139): Connect 0x6a675bd6 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:53:40,007 DEBUG [Listener at localhost/42557] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@629018b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:53:40,019 DEBUG [hconnection-0x6392fb1b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:53:40,034 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41674, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:53:40,047 INFO [Listener at localhost/42557] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33303,1675936414925 2023-02-09 09:53:40,048 DEBUG [Listener at localhost/42557] zookeeper.ReadOnlyZKClient(139): Connect 0x6f46f74d to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:53:40,052 DEBUG [Listener at localhost/42557] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7defec53, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:53:40,053 INFO [Listener at localhost/42557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 09:53:40,100 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=210, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=328, ProcessCount=168, AvailableMemoryMB=4290 2023-02-09 09:53:40,105 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:40,105 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:40,112 DEBUG [Listener at localhost/42557] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:53:40,115 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56486, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:53:40,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 09:53:40,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:40,130 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:53:40,133 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:53:40,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-02-09 09:53:40,135 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:40,136 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455 empty. 2023-02-09 09:53:40,138 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:40,138 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-02-09 09:53:40,141 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:40,141 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:40,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 09:53:40,247 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:40,247 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:40,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 09:53:40,449 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:40,449 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:40,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 09:53:40,556 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-09 09:53:40,558 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1a102fc8ca7411175545f4844f76a455, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:53:40,751 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:40,752 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:40,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 09:53:40,971 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:40,971 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1603): Closing 1a102fc8ca7411175545f4844f76a455, disabling compactions & flushes 2023-02-09 09:53:40,972 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:40,972 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:40,972 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. after waiting 0 ms 2023-02-09 09:53:40,972 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:40,972 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:40,972 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1557): Region close journal for 1a102fc8ca7411175545f4844f76a455: Waiting for close lock at 1675936420971Disabling compacts and flushes for region at 1675936420971Disabling writes for close at 1675936420972 (+1 ms)Writing region close event to WAL at 1675936420972Closed at 1675936420972 2023-02-09 09:53:40,976 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:53:40,977 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675936420977"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936420977"}]},"ts":"1675936420977"} 2023-02-09 09:53:40,980 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 09:53:40,981 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:53:40,982 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936420981"}]},"ts":"1675936420981"} 2023-02-09 09:53:40,983 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-02-09 09:53:40,988 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1a102fc8ca7411175545f4844f76a455, ASSIGN}] 2023-02-09 09:53:40,990 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1a102fc8ca7411175545f4844f76a455, ASSIGN 2023-02-09 09:53:40,992 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1a102fc8ca7411175545f4844f76a455, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:53:41,143 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1a102fc8ca7411175545f4844f76a455, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:41,144 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675936421143"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936421143"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936421143"}]},"ts":"1675936421143"} 2023-02-09 09:53:41,147 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 1a102fc8ca7411175545f4844f76a455, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:53:41,254 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:41,254 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:41,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 09:53:41,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:41,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1a102fc8ca7411175545f4844f76a455, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:53:41,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:41,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:41,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:41,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:41,310 INFO [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:41,313 DEBUG [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1 2023-02-09 09:53:41,313 DEBUG [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1 2023-02-09 09:53:41,316 INFO [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1a102fc8ca7411175545f4844f76a455 columnFamilyName family1 2023-02-09 09:53:41,319 INFO [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] regionserver.HStore(310): Store=1a102fc8ca7411175545f4844f76a455/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:41,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:41,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:41,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:41,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:53:41,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1a102fc8ca7411175545f4844f76a455; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62039983, jitterRate=-0.0755322128534317}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:53:41,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1a102fc8ca7411175545f4844f76a455: Running coprocessor pre-open hook at 1675936421308Writing region info on filesystem at 1675936421308Initializing all the Stores at 1675936421309 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675936421309Cleaning up temporary data from old regions at 1675936421323 (+14 ms)Cleaning up detritus from prior splits at 1675936421323Running coprocessor post-open hooks at 1675936421333 (+10 ms)Region opened successfully at 1675936421333 2023-02-09 09:53:41,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455., pid=11, masterSystemTime=1675936421301 2023-02-09 09:53:41,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:41,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:41,340 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1a102fc8ca7411175545f4844f76a455, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:41,341 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675936421340"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936421340"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936421340"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936421340"}]},"ts":"1675936421340"} 2023-02-09 09:53:41,348 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-09 09:53:41,348 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 1a102fc8ca7411175545f4844f76a455, server=jenkins-hbase4.apache.org,42719,1675936416061 in 198 msec 2023-02-09 09:53:41,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-09 09:53:41,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1a102fc8ca7411175545f4844f76a455, ASSIGN in 360 msec 2023-02-09 09:53:41,353 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:53:41,353 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936421353"}]},"ts":"1675936421353"} 2023-02-09 09:53:41,356 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-02-09 09:53:41,359 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:53:41,361 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.2320 sec 2023-02-09 09:53:42,257 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:42,257 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:42,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 09:53:42,259 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-02-09 09:53:42,277 DEBUG [hconnection-0x462145a9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:53:42,281 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41690, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:53:42,435 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:42,435 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:42,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:42,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:42,457 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:42,457 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:42,457 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 09:53:42,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 09:53:42,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:42,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:42,460 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:42,460 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:42,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:42,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:42,461 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:42,461 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:42,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:42,461 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:42,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:42,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:42,462 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:42,464 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:42,464 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 09:53:42,464 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:42,465 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 09:53:42,465 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:42,465 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:42,466 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:42,467 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. started... 2023-02-09 09:53:42,467 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1a102fc8ca7411175545f4844f76a455 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 09:53:43,020 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113052f32828fce1494582604e4f3b3a0b28 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113052f32828fce1494582604e4f3b3a0b28 2023-02-09 09:53:43,035 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/5e77753fd4944c1ba4fc9917644b08a5 2023-02-09 09:53:43,045 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/5e77753fd4944c1ba4fc9917644b08a5 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5e77753fd4944c1ba4fc9917644b08a5 2023-02-09 09:53:43,054 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5e77753fd4944c1ba4fc9917644b08a5, entries=1000, sequenceid=5, filesize=175.8 K 2023-02-09 09:53:43,057 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 1a102fc8ca7411175545f4844f76a455 in 590ms, sequenceid=5, compaction requested=false 2023-02-09 09:53:43,058 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:43,061 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:43,061 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:43,061 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:43,062 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:43,062 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,062 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 09:53:43,062 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 09:53:43,065 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,065 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:43,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:43,065 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,066 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:43,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:43,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:43,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:43,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 09:53:43,068 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 09:53:43,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f317ddf[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:43,068 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,070 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,071 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 09:53:43,071 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 09:53:43,071 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,071 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:43,071 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:43,075 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 09:53:43,075 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,075 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:43,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:43,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:43,075 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 09:53:43,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:43,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:43,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:43,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:43,079 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 09:53:43,079 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:43,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 09:53:43,080 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:43,080 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,080 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:43,082 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,082 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,082 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:43,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:43,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:43,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:43,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:43,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:43,083 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:43,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,086 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:43,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,094 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:43,094 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:43,094 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:43,094 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:43,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:43,094 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,094 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:43,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:43,094 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,095 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,095 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,095 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:43,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:43,096 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:43,097 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:43,197 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:43,198 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:43,198 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:43,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:43,215 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:43,215 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:43,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:43,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:43,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,220 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:43,220 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:43,220 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 09:53:43,220 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 09:53:43,221 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,221 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,223 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:43,223 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:43,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:43,223 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,223 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:43,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:43,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,225 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 09:53:43,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,225 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 09:53:43,225 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:43,226 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 09:53:43,226 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:43,226 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:43,226 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:43,226 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. started... 2023-02-09 09:53:43,227 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1a102fc8ca7411175545f4844f76a455 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 09:53:43,260 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112814fca2afdba2430f97393f810b61a20c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112814fca2afdba2430f97393f810b61a20c 2023-02-09 09:53:43,279 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/6aff970039854c32bf277ecfd33e0d30 2023-02-09 09:53:43,289 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/6aff970039854c32bf277ecfd33e0d30 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/6aff970039854c32bf277ecfd33e0d30 2023-02-09 09:53:43,298 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/6aff970039854c32bf277ecfd33e0d30, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 09:53:43,300 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 1a102fc8ca7411175545f4844f76a455 in 73ms, sequenceid=9, compaction requested=false 2023-02-09 09:53:43,301 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:43,301 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:43,301 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:43,302 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:43,302 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,302 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 09:53:43,302 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 09:53:43,305 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,305 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:43,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:43,306 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,306 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:43,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:43,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:43,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:43,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 09:53:43,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e9e2a1d[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:43,309 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 09:53:43,309 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,310 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,311 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 09:53:43,311 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 09:53:43,311 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:43,311 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,311 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:43,313 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 09:53:43,313 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,313 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:43,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:43,313 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 09:53:43,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:43,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:43,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:43,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:43,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:43,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 09:53:43,318 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 09:53:43,318 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:43,318 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:43,318 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,318 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:43,325 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,325 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:43,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,325 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:43,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:43,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:43,327 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,329 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:43,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,334 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,334 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:43,334 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:43,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:43,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:43,334 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,334 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,334 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,334 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:43,334 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,335 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,335 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:43,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:43,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:43,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:43,336 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:43,336 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:43,436 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:43,436 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:43,437 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:43,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:43,448 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:43,448 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:43,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:43,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:43,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,456 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:43,456 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:43,457 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 09:53:43,457 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 09:53:43,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,459 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,459 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:43,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:43,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:43,460 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,460 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:43,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,461 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:43,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,461 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,462 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 09:53:43,462 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,462 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 09:53:43,462 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:43,463 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 09:53:43,463 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:43,463 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:43,463 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:43,463 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. started... 2023-02-09 09:53:43,463 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1a102fc8ca7411175545f4844f76a455 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:43,504 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201565ede70bf8f433ca3e7c0943d018919 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201565ede70bf8f433ca3e7c0943d018919 2023-02-09 09:53:43,527 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/cdef09e75b3e4fc4b4f0c99ddcd648d2 2023-02-09 09:53:43,539 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/cdef09e75b3e4fc4b4f0c99ddcd648d2 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/cdef09e75b3e4fc4b4f0c99ddcd648d2 2023-02-09 09:53:43,550 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/cdef09e75b3e4fc4b4f0c99ddcd648d2, entries=1, sequenceid=13, filesize=5.0 K 2023-02-09 09:53:43,552 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1a102fc8ca7411175545f4844f76a455 in 89ms, sequenceid=13, compaction requested=true 2023-02-09 09:53:43,552 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:43,552 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:43,552 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:43,553 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:43,553 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,553 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 09:53:43,553 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 09:53:43,555 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,555 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:43,556 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,556 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:43,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 09:53:43,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ef5a049[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:43,559 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 09:53:43,559 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,561 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,562 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 09:53:43,562 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 09:53:43,562 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:43,562 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,562 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:43,564 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 09:53:43,565 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:43,565 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 09:53:43,568 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:43,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:43,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:43,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:43,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 09:53:43,573 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 09:53:43,573 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:43,573 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:43,573 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,573 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:43,575 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,575 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:43,575 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:43,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:43,577 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:43,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:43,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:43,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:43,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,579 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:43,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,586 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,586 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:43,587 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:43,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:43,587 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:43,587 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,587 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:43,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:43,587 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,587 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:43,587 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,588 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,588 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:43,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 09:53:43,588 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:43,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:43,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:43,594 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-09 09:53:43,596 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-09 09:53:43,688 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:43,689 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:43,689 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:43,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:43,697 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:43,697 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:43,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:43,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:43,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,701 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:43,701 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:43,702 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 09:53:43,702 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 09:53:43,702 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,702 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,704 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,704 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:43,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:43,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:43,705 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:43,705 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:43,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:43,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 09:53:43,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:43,706 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 09:53:43,706 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:43,707 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 09:53:43,707 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:43,707 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:43,707 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:43,707 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. started... 2023-02-09 09:53:43,707 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1a102fc8ca7411175545f4844f76a455 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:43,740 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120564e1bd948b33485a9fc1d6ebefe87fcd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120564e1bd948b33485a9fc1d6ebefe87fcd 2023-02-09 09:53:44,161 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/b406fc2c08494dd48e5d14571db0ec6c 2023-02-09 09:53:44,171 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/b406fc2c08494dd48e5d14571db0ec6c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/b406fc2c08494dd48e5d14571db0ec6c 2023-02-09 09:53:44,181 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/b406fc2c08494dd48e5d14571db0ec6c, entries=1, sequenceid=17, filesize=5.0 K 2023-02-09 09:53:44,182 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1a102fc8ca7411175545f4844f76a455 in 475ms, sequenceid=17, compaction requested=true 2023-02-09 09:53:44,182 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:44,182 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:44,183 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:44,183 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:44,183 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,183 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 09:53:44,183 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 09:53:44,185 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,185 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:44,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:44,186 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,186 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:44,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:44,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:44,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:44,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 09:53:44,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d1563f3[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:44,188 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 09:53:44,188 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,189 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,190 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 09:53:44,190 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 09:53:44,190 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:44,190 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,190 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:44,192 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 09:53:44,192 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:44,192 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,192 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 09:53:44,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:44,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:44,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 09:53:44,196 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 09:53:44,196 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:44,196 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:44,196 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,196 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:44,199 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,199 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:44,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:44,199 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:44,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,200 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:44,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:44,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:44,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:44,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,202 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:44,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,207 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,207 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,207 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,207 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:44,207 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,207 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:44,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:44,207 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 09:53:44,207 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,207 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:44,207 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:44,207 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:44,207 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:44,208 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:44,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:44,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:44,208 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:44,289 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:53:44,309 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:44,309 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:44,309 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:44,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:44,318 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:44,318 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:44,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:44,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:44,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,322 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:44,322 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:44,322 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 09:53:44,322 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 09:53:44,323 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,323 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,324 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:44,324 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:44,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:44,325 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:44,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,325 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:44,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:44,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,326 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 09:53:44,326 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:44,326 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 09:53:44,326 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 09:53:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:44,327 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:44,327 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. started... 2023-02-09 09:53:44,328 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1a102fc8ca7411175545f4844f76a455 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:44,755 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228e33584c6983349f49681aac8819c67db to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228e33584c6983349f49681aac8819c67db 2023-02-09 09:53:45,077 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 09:53:45,078 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-09 09:53:45,168 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/1340238962f14b97bc4fc1a849c92d1c 2023-02-09 09:53:45,177 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/1340238962f14b97bc4fc1a849c92d1c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1340238962f14b97bc4fc1a849c92d1c 2023-02-09 09:53:45,185 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1340238962f14b97bc4fc1a849c92d1c, entries=1, sequenceid=21, filesize=5.0 K 2023-02-09 09:53:45,186 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1a102fc8ca7411175545f4844f76a455 in 858ms, sequenceid=21, compaction requested=true 2023-02-09 09:53:45,186 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:45,186 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 09:53:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 09:53:45,192 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,192 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:45,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:45,192 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,192 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:45,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:45,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:45,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:45,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 09:53:45,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@798e2e0b[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:45,194 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 09:53:45,194 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,196 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,196 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 09:53:45,196 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 09:53:45,196 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:45,196 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,196 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:45,197 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 09:53:45,197 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,197 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:45,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:45,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:45,197 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 09:53:45,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:45,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:45,200 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 09:53:45,200 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 09:53:45,200 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:45,201 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:45,201 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,201 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:45,202 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,202 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:45,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,202 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:45,203 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,204 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:45,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,212 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:45,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,213 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:45,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:45,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:45,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:45,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 09:53:45,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:45,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:45,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:45,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:45,214 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:45,214 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:45,314 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:45,314 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:45,314 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:45,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:45,322 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:45,322 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:45,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:45,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:45,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,325 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:45,325 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:45,326 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 09:53:45,326 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 09:53:45,326 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,327 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,328 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:45,328 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:45,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:45,328 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:45,328 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:45,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,329 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,329 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 09:53:45,329 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:45,329 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 09:53:45,330 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:45,330 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 09:53:45,330 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:45,330 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:45,330 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:45,330 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. started... 2023-02-09 09:53:45,331 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1a102fc8ca7411175545f4844f76a455 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:45,754 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123143938daeccef44db994c680c67107acc to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123143938daeccef44db994c680c67107acc 2023-02-09 09:53:46,164 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/1a7fe1b736cc44f4ad80dd08bae07495 2023-02-09 09:53:46,173 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/1a7fe1b736cc44f4ad80dd08bae07495 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1a7fe1b736cc44f4ad80dd08bae07495 2023-02-09 09:53:46,180 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1a7fe1b736cc44f4ad80dd08bae07495, entries=1, sequenceid=25, filesize=5.0 K 2023-02-09 09:53:46,182 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1a102fc8ca7411175545f4844f76a455 in 851ms, sequenceid=25, compaction requested=true 2023-02-09 09:53:46,182 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:46,182 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:46,182 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:46,183 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:46,183 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,183 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 09:53:46,183 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 09:53:46,185 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,185 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:46,185 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,185 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:46,187 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 09:53:46,187 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@764f3de5[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:46,187 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 09:53:46,187 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,188 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,189 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 09:53:46,189 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 09:53:46,189 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:46,189 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,189 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:46,191 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,191 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 09:53:46,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:46,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:46,191 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:46,191 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 09:53:46,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:46,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:46,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:46,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:46,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 09:53:46,194 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 09:53:46,194 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:46,194 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:46,194 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,194 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:46,195 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,195 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:46,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:46,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,195 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:46,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:46,196 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:46,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:46,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,197 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,200 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,200 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:46,200 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,200 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:46,200 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 09:53:46,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:46,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:46,200 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:46,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:46,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:46,200 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,201 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,201 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,201 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,201 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:46,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:46,201 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:46,202 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:46,302 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:46,302 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:46,302 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:46,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:46,310 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:46,310 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:46,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:46,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:46,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,314 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:46,314 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:46,314 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 09:53:46,314 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 09:53:46,315 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,315 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,316 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:46,316 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:46,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:46,317 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:46,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,317 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:46,317 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:46,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,318 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,318 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 09:53:46,318 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:46,318 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 09:53:46,318 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:46,319 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 09:53:46,324 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:46,324 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:46,324 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:46,324 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. started... 2023-02-09 09:53:46,324 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1a102fc8ca7411175545f4844f76a455 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:46,745 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601019fae6dd9217f435a95844040dd03c332 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601019fae6dd9217f435a95844040dd03c332 2023-02-09 09:53:47,179 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/70c1a58900664a548aee4be2576f78fd 2023-02-09 09:53:47,187 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/70c1a58900664a548aee4be2576f78fd as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/70c1a58900664a548aee4be2576f78fd 2023-02-09 09:53:47,195 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/70c1a58900664a548aee4be2576f78fd, entries=1, sequenceid=29, filesize=5.0 K 2023-02-09 09:53:47,196 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1a102fc8ca7411175545f4844f76a455 in 872ms, sequenceid=29, compaction requested=true 2023-02-09 09:53:47,197 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:47,197 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:47,197 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:47,197 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:47,197 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,197 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 09:53:47,197 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 09:53:47,200 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,200 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:47,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:47,201 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,201 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:47,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:47,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:47,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:47,203 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 09:53:47,203 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@357b3c8d[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:47,203 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 09:53:47,203 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,204 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,204 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 09:53:47,204 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 09:53:47,204 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:47,204 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,204 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:47,206 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 09:53:47,206 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:47,206 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,206 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 09:53:47,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:47,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:47,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:47,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:47,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:47,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:47,209 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 09:53:47,209 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 09:53:47,209 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:47,209 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:47,209 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,209 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:47,211 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,211 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:47,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:47,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,211 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:47,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:47,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:47,211 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:47,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:47,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,213 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:47,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,219 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,219 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,219 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:47,219 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,219 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:47,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 09:53:47,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:47,219 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:47,219 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:47,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:47,220 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,220 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:47,220 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,220 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:47,220 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:47,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:47,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:47,320 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:47,321 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:47,321 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:47,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:47,329 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:47,329 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:47,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:47,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:47,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,333 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:47,333 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:47,333 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 09:53:47,333 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 09:53:47,334 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,334 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,335 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,335 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:47,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:47,336 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,336 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,336 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:47,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,337 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 09:53:47,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,337 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 09:53:47,337 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:47,337 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 09:53:47,338 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:47,338 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:47,338 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:47,338 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. started... 2023-02-09 09:53:47,338 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1a102fc8ca7411175545f4844f76a455 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:47,759 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010300dc96da5b7b415bb51ba5c24d9c83a7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010300dc96da5b7b415bb51ba5c24d9c83a7 2023-02-09 09:53:47,768 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/193e60cc03cb4f0c8e3467769075660d 2023-02-09 09:53:47,775 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/193e60cc03cb4f0c8e3467769075660d as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/193e60cc03cb4f0c8e3467769075660d 2023-02-09 09:53:47,782 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/193e60cc03cb4f0c8e3467769075660d, entries=1, sequenceid=33, filesize=5.0 K 2023-02-09 09:53:47,783 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1a102fc8ca7411175545f4844f76a455 in 445ms, sequenceid=33, compaction requested=true 2023-02-09 09:53:47,784 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:47,784 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:47,784 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:47,784 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:47,784 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,784 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 09:53:47,784 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 09:53:47,787 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,787 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:47,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:47,787 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,787 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:47,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:47,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:47,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:47,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 09:53:47,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2367cb2a[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:47,789 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 09:53:47,789 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,790 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,790 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 09:53:47,790 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 09:53:47,790 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:47,790 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,790 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:47,792 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 09:53:47,792 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,792 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:47,792 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 09:53:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:47,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 09:53:47,795 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 09:53:47,795 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:47,795 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:47,795 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,795 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:47,796 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,796 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,796 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:47,797 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:47,798 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:47,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,802 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,802 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,802 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:47,802 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:47,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:47,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:47,802 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:47,802 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,802 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:47,802 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:47,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 09:53:47,803 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:47,802 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:47,803 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,803 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:47,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:47,903 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:47,904 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:47,904 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:47,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:47,912 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:47,912 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:47,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:47,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:47,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,915 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:47,915 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:47,916 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 09:53:47,916 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 09:53:47,916 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,916 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,917 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,917 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:47,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:47,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:47,918 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:47,918 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:47,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,918 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:47,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,919 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,919 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 09:53:47,919 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:47,920 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 09:53:47,920 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:47,920 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 09:53:47,921 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:47,921 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:47,921 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:47,921 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. started... 2023-02-09 09:53:47,921 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1a102fc8ca7411175545f4844f76a455 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:48,341 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509071181de809a7b45009c1c89ff40c85b28 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509071181de809a7b45009c1c89ff40c85b28 2023-02-09 09:53:48,750 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/c994d3d655c34a32b7e2ae78e31fd411 2023-02-09 09:53:48,757 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/c994d3d655c34a32b7e2ae78e31fd411 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/c994d3d655c34a32b7e2ae78e31fd411 2023-02-09 09:53:48,764 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/c994d3d655c34a32b7e2ae78e31fd411, entries=1, sequenceid=37, filesize=5.0 K 2023-02-09 09:53:48,765 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1a102fc8ca7411175545f4844f76a455 in 844ms, sequenceid=37, compaction requested=true 2023-02-09 09:53:48,765 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:48,765 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:48,765 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:48,766 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:48,766 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,766 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 09:53:48,766 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 09:53:48,768 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,768 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:48,768 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:48,768 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:48,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:48,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 09:53:48,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b11473b[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:48,769 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 09:53:48,770 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,771 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 09:53:48,771 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,771 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 09:53:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:48,776 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 09:53:48,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,776 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:48,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:48,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:48,777 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 09:53:48,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:48,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:48,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:48,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:48,780 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 09:53:48,780 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 09:53:48,780 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:48,780 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:48,780 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,780 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:48,781 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,781 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:48,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:48,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,781 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:48,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:48,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:48,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:48,782 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:48,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,783 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:48,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,786 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,786 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,786 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:48,786 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,786 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:48,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 09:53:48,786 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:48,787 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:48,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:48,786 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:48,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:48,787 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,787 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,787 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,787 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:48,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:48,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:48,787 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:48,887 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:48,888 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:48,888 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:48,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:48,895 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:48,896 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:48,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:48,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:48,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,899 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:48,899 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:48,899 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 09:53:48,899 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 09:53:48,900 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,900 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,901 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,901 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:48,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:48,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:48,901 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:48,902 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:48,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,902 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:48,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,903 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,903 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 09:53:48,903 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:48,903 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 09:53:48,903 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:48,904 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 09:53:48,905 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:48,905 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:48,905 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:48,905 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. started... 2023-02-09 09:53:48,905 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1a102fc8ca7411175545f4844f76a455 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:49,326 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e3958808be0345589948263cb6b0d6f4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120e3958808be0345589948263cb6b0d6f4 2023-02-09 09:53:49,734 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/914abd1aa71147448328b1f0c48c3e05 2023-02-09 09:53:49,742 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/914abd1aa71147448328b1f0c48c3e05 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/914abd1aa71147448328b1f0c48c3e05 2023-02-09 09:53:49,748 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/914abd1aa71147448328b1f0c48c3e05, entries=1, sequenceid=41, filesize=5.0 K 2023-02-09 09:53:49,750 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1a102fc8ca7411175545f4844f76a455 in 845ms, sequenceid=41, compaction requested=true 2023-02-09 09:53:49,750 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:49,750 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:49,750 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:49,750 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:49,750 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,751 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 09:53:49,751 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 09:53:49,753 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,753 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:49,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:49,753 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,753 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:49,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:49,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:49,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:49,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 09:53:49,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@243ee7a9[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:49,755 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 09:53:49,755 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,756 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,756 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 09:53:49,756 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 09:53:49,756 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:49,756 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,756 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:49,760 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,760 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 09:53:49,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:49,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:49,760 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:49,760 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 09:53:49,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:49,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:49,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:49,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:49,764 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 09:53:49,764 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 09:53:49,764 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:49,764 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:49,764 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,764 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:49,766 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,766 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:49,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:49,767 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:49,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:49,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:49,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:49,767 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:49,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,768 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:49,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,771 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,771 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:49,771 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 09:53:49,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:49,771 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 09:53:49,771 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:49,771 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:49,771 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:49,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:49,771 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:49,771 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,772 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,772 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:49,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:49,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:49,772 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:49,772 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,872 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:49,872 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:49,872 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:49,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:49,878 DEBUG [Listener at localhost/42557] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 09:53:49,881 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45876, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 09:53:49,883 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:49,896 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 09:53:49,897 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 09:53:49,898 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:53:49,898 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937029898 2023-02-09 09:53:49,900 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:53:49,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-09 09:53:49,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 09:53:49,904 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x6fd61940 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:53:49,909 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4b2cb6ce, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:53:49,910 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:49,975 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ccfcc9fdb1004d4fb66369dbc1853444 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103ccfcc9fdb1004d4fb66369dbc1853444 2023-02-09 09:53:49,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 09:53:49,994 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:53:49,996 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50082, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:53:50,006 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:50,006 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:50,007 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:53:50,009 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45880, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:53:50,041 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/05f948d4dfc94548913007e4f554bf22 first=Optional[r3] last=Optional[r6] 2023-02-09 09:53:50,053 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/05f948d4dfc94548913007e4f554bf22}] 2023-02-09 09:53:50,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/05f948d4dfc94548913007e4f554bf22 for inclusion in 1a102fc8ca7411175545f4844f76a455/family1 2023-02-09 09:53:50,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 09:53:50,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:53:50,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:50,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/05f948d4dfc94548913007e4f554bf22 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__6jlru5topmjrmcv4d17c2o4nbfgihg180v7h94na8p9r6agmhutctht161n92f9c/family1/05f948d4dfc94548913007e4f554bf22 2023-02-09 09:53:50,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__6jlru5topmjrmcv4d17c2o4nbfgihg180v7h94na8p9r6agmhutctht161n92f9c/family1/05f948d4dfc94548913007e4f554bf22 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5030149460234dbdadfd4333c9b417ff_SeqId_44_ 2023-02-09 09:53:50,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__6jlru5topmjrmcv4d17c2o4nbfgihg180v7h94na8p9r6agmhutctht161n92f9c/family1/05f948d4dfc94548913007e4f554bf22 into 1a102fc8ca7411175545f4844f76a455/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5030149460234dbdadfd4333c9b417ff_SeqId_44_ - updating store file list. 2023-02-09 09:53:50,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5030149460234dbdadfd4333c9b417ff_SeqId_44_ into 1a102fc8ca7411175545f4844f76a455/family1 2023-02-09 09:53:50,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__6jlru5topmjrmcv4d17c2o4nbfgihg180v7h94na8p9r6agmhutctht161n92f9c/family1/05f948d4dfc94548913007e4f554bf22 into 1a102fc8ca7411175545f4844f76a455/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5030149460234dbdadfd4333c9b417ff_SeqId_44_) 2023-02-09 09:53:50,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__6jlru5topmjrmcv4d17c2o4nbfgihg180v7h94na8p9r6agmhutctht161n92f9c/family1/05f948d4dfc94548913007e4f554bf22 2023-02-09 09:53:50,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 09:53:50,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 09:53:50,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): 1a102fc8ca7411175545f4844f76a455/family1 is initiating minor compaction 2023-02-09 09:53:50,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7e5cafe0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:53:50,101 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1a102fc8ca7411175545f4844f76a455/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:50,102 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/6aff970039854c32bf277ecfd33e0d30, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/cdef09e75b3e4fc4b4f0c99ddcd648d2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/b406fc2c08494dd48e5d14571db0ec6c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1340238962f14b97bc4fc1a849c92d1c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1a7fe1b736cc44f4ad80dd08bae07495, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/70c1a58900664a548aee4be2576f78fd, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/193e60cc03cb4f0c8e3467769075660d, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/c994d3d655c34a32b7e2ae78e31fd411, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/914abd1aa71147448328b1f0c48c3e05, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5030149460234dbdadfd4333c9b417ff_SeqId_44_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp, totalSize=50.2 K 2023-02-09 09:53:50,102 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 6aff970039854c32bf277ecfd33e0d30, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 09:53:50,102 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting cdef09e75b3e4fc4b4f0c99ddcd648d2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 09:53:50,102 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting b406fc2c08494dd48e5d14571db0ec6c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 09:53:50,102 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 1340238962f14b97bc4fc1a849c92d1c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 09:53:50,102 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 1a7fe1b736cc44f4ad80dd08bae07495, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 09:53:50,102 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 70c1a58900664a548aee4be2576f78fd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 09:53:50,102 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 193e60cc03cb4f0c8e3467769075660d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 09:53:50,103 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting c994d3d655c34a32b7e2ae78e31fd411, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 09:53:50,103 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 914abd1aa71147448328b1f0c48c3e05, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 09:53:50,103 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 5030149460234dbdadfd4333c9b417ff_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 09:53:50,117 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 09:53:50,119 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:53:50,125 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228e33584c6983349f49681aac8819c67db to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228e33584c6983349f49681aac8819c67db 2023-02-09 09:53:50,127 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123143938daeccef44db994c680c67107acc to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123143938daeccef44db994c680c67107acc 2023-02-09 09:53:50,130 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601019fae6dd9217f435a95844040dd03c332 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601019fae6dd9217f435a95844040dd03c332 2023-02-09 09:53:50,133 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010300dc96da5b7b415bb51ba5c24d9c83a7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010300dc96da5b7b415bb51ba5c24d9c83a7 2023-02-09 09:53:50,133 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-02-09 09:53:50,133 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 09:53:50,133 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:50,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:50,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:50,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:50,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:50,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:50,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:50,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:50,155 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1a102fc8ca7411175545f4844f76a455#family1#compaction#10 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:53:50,183 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/f98b793d47dc41ecbcc50dc826be87e6 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/f98b793d47dc41ecbcc50dc826be87e6 2023-02-09 09:53:50,205 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 1a102fc8ca7411175545f4844f76a455/family1 of 1a102fc8ca7411175545f4844f76a455 into f98b793d47dc41ecbcc50dc826be87e6(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:53:50,205 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:50,207 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1675936430098; duration=0sec 2023-02-09 09:53:50,208 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:53:50,235 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/6aff970039854c32bf277ecfd33e0d30, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/cdef09e75b3e4fc4b4f0c99ddcd648d2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/b406fc2c08494dd48e5d14571db0ec6c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1340238962f14b97bc4fc1a849c92d1c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1a7fe1b736cc44f4ad80dd08bae07495, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/70c1a58900664a548aee4be2576f78fd, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/193e60cc03cb4f0c8e3467769075660d, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/c994d3d655c34a32b7e2ae78e31fd411, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/914abd1aa71147448328b1f0c48c3e05, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5030149460234dbdadfd4333c9b417ff_SeqId_44_] to archive 2023-02-09 09:53:50,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:53:50,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/6aff970039854c32bf277ecfd33e0d30 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/6aff970039854c32bf277ecfd33e0d30 2023-02-09 09:53:50,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/cdef09e75b3e4fc4b4f0c99ddcd648d2 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/cdef09e75b3e4fc4b4f0c99ddcd648d2 2023-02-09 09:53:50,243 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/b406fc2c08494dd48e5d14571db0ec6c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/b406fc2c08494dd48e5d14571db0ec6c 2023-02-09 09:53:50,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1340238962f14b97bc4fc1a849c92d1c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1340238962f14b97bc4fc1a849c92d1c 2023-02-09 09:53:50,246 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1a7fe1b736cc44f4ad80dd08bae07495 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1a7fe1b736cc44f4ad80dd08bae07495 2023-02-09 09:53:50,247 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/70c1a58900664a548aee4be2576f78fd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/70c1a58900664a548aee4be2576f78fd 2023-02-09 09:53:50,249 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/193e60cc03cb4f0c8e3467769075660d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/193e60cc03cb4f0c8e3467769075660d 2023-02-09 09:53:50,251 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/c994d3d655c34a32b7e2ae78e31fd411 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/c994d3d655c34a32b7e2ae78e31fd411 2023-02-09 09:53:50,252 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/914abd1aa71147448328b1f0c48c3e05 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/914abd1aa71147448328b1f0c48c3e05 2023-02-09 09:53:50,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5030149460234dbdadfd4333c9b417ff_SeqId_44_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5030149460234dbdadfd4333c9b417ff_SeqId_44_ 2023-02-09 09:53:51,028 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056fc26115626b47e0aaa54e87b42a3d7e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512056fc26115626b47e0aaa54e87b42a3d7e 2023-02-09 09:53:51,030 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 09:53:51,050 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:51,050 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:51,061 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f6290cf5b764192aeef29b40696301e first=Optional[r00] last=Optional[r2] 2023-02-09 09:53:51,062 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f6290cf5b764192aeef29b40696301e}] 2023-02-09 09:53:51,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f6290cf5b764192aeef29b40696301e for inclusion in 1a102fc8ca7411175545f4844f76a455/family1 2023-02-09 09:53:51,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 09:53:51,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:53:51,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:51,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f6290cf5b764192aeef29b40696301e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hqv84ehv2mjt160r0fab8q6dl9sf19fgvnqq3t34qv6qmcd2gfdq80f7kt5m0oiq/family1/7f6290cf5b764192aeef29b40696301e 2023-02-09 09:53:51,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hqv84ehv2mjt160r0fab8q6dl9sf19fgvnqq3t34qv6qmcd2gfdq80f7kt5m0oiq/family1/7f6290cf5b764192aeef29b40696301e as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1101c35f134e46488a12d40317e164c1_SeqId_47_ 2023-02-09 09:53:51,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hqv84ehv2mjt160r0fab8q6dl9sf19fgvnqq3t34qv6qmcd2gfdq80f7kt5m0oiq/family1/7f6290cf5b764192aeef29b40696301e into 1a102fc8ca7411175545f4844f76a455/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1101c35f134e46488a12d40317e164c1_SeqId_47_ - updating store file list. 2023-02-09 09:53:51,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1101c35f134e46488a12d40317e164c1_SeqId_47_ into 1a102fc8ca7411175545f4844f76a455/family1 2023-02-09 09:53:51,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hqv84ehv2mjt160r0fab8q6dl9sf19fgvnqq3t34qv6qmcd2gfdq80f7kt5m0oiq/family1/7f6290cf5b764192aeef29b40696301e into 1a102fc8ca7411175545f4844f76a455/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1101c35f134e46488a12d40317e164c1_SeqId_47_) 2023-02-09 09:53:51,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hqv84ehv2mjt160r0fab8q6dl9sf19fgvnqq3t34qv6qmcd2gfdq80f7kt5m0oiq/family1/7f6290cf5b764192aeef29b40696301e 2023-02-09 09:53:51,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 09:53:51,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 09:53:51,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): 1a102fc8ca7411175545f4844f76a455/family1 is initiating minor compaction (all files) 2023-02-09 09:53:51,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2856eed0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:53:51,081 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1a102fc8ca7411175545f4844f76a455/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:51,083 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5e77753fd4944c1ba4fc9917644b08a5, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/f98b793d47dc41ecbcc50dc826be87e6, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1101c35f134e46488a12d40317e164c1_SeqId_47_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp, totalSize=358.6 K 2023-02-09 09:53:51,083 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 5e77753fd4944c1ba4fc9917644b08a5, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840054 2023-02-09 09:53:51,084 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting f98b793d47dc41ecbcc50dc826be87e6, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240054 2023-02-09 09:53:51,085 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 1101c35f134e46488a12d40317e164c1_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840054 2023-02-09 09:53:51,085 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 09:53:51,086 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:53:51,088 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113052f32828fce1494582604e4f3b3a0b28 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113052f32828fce1494582604e4f3b3a0b28 2023-02-09 09:53:51,090 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201565ede70bf8f433ca3e7c0943d018919 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201565ede70bf8f433ca3e7c0943d018919 2023-02-09 09:53:51,092 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120564e1bd948b33485a9fc1d6ebefe87fcd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120564e1bd948b33485a9fc1d6ebefe87fcd 2023-02-09 09:53:51,092 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-02-09 09:53:51,092 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 09:53:51,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:51,092 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 09:53:51,092 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:53:51,093 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 09:53:51,095 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.2050 sec 2023-02-09 09:53:51,120 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1a102fc8ca7411175545f4844f76a455#family1#compaction#11 average throughput is 7.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:53:51,240 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:51,240 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:51,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 09:53:51,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:51,672 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-09 09:53:51,673 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-09 09:53:51,673 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-09 09:53:51,678 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-02-09 09:53:51,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1a102fc8ca7411175545f4844f76a455, REOPEN/MOVE}] 2023-02-09 09:53:51,691 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1a102fc8ca7411175545f4844f76a455, REOPEN/MOVE 2023-02-09 09:53:51,692 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1a102fc8ca7411175545f4844f76a455, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:51,692 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675936431692"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936431692"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936431692"}]},"ts":"1675936431692"} 2023-02-09 09:53:51,695 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 1a102fc8ca7411175545f4844f76a455, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:53:51,853 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:51,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1a102fc8ca7411175545f4844f76a455, disabling compactions & flushes 2023-02-09 09:53:51,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:52,360 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/.tmp/family1/9bfc8cb02c7e4662ae039f7c2166d052 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/9bfc8cb02c7e4662ae039f7c2166d052 2023-02-09 09:53:52,368 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1a102fc8ca7411175545f4844f76a455/family1 of 1a102fc8ca7411175545f4844f76a455 into 9bfc8cb02c7e4662ae039f7c2166d052(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:53:52,368 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:52,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:52,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:52,368 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1675936431081; duration=1sec 2023-02-09 09:53:52,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. after waiting 0 ms 2023-02-09 09:53:52,368 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:53:52,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:52,371 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5e77753fd4944c1ba4fc9917644b08a5, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/f98b793d47dc41ecbcc50dc826be87e6, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1101c35f134e46488a12d40317e164c1_SeqId_47_] to archive 2023-02-09 09:53:52,372 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:53:52,374 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5e77753fd4944c1ba4fc9917644b08a5 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/5e77753fd4944c1ba4fc9917644b08a5 2023-02-09 09:53:52,376 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/f98b793d47dc41ecbcc50dc826be87e6 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/f98b793d47dc41ecbcc50dc826be87e6 2023-02-09 09:53:52,378 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1101c35f134e46488a12d40317e164c1_SeqId_47_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/1101c35f134e46488a12d40317e164c1_SeqId_47_ 2023-02-09 09:53:52,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 09:53:52,385 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:52,385 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1a102fc8ca7411175545f4844f76a455: Waiting for close lock at 1675936431853Running coprocessor pre-close hooks at 1675936431853Disabling compacts and flushes for region at 1675936431853Disabling writes for close at 1675936432368 (+515 ms)Writing region close event to WAL at 1675936432381 (+13 ms)Running coprocessor post-close hooks at 1675936432385 (+4 ms)Closed at 1675936432385 2023-02-09 09:53:52,385 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 1a102fc8ca7411175545f4844f76a455 to self. 2023-02-09 09:53:52,387 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:52,388 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1a102fc8ca7411175545f4844f76a455, regionState=CLOSED 2023-02-09 09:53:52,388 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675936432388"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936432388"}]},"ts":"1675936432388"} 2023-02-09 09:53:52,393 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-09 09:53:52,393 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 1a102fc8ca7411175545f4844f76a455, server=jenkins-hbase4.apache.org,42719,1675936416061 in 695 msec 2023-02-09 09:53:52,394 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1a102fc8ca7411175545f4844f76a455, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=true 2023-02-09 09:53:52,544 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1a102fc8ca7411175545f4844f76a455, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:52,544 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675936432544"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936432544"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936432544"}]},"ts":"1675936432544"} 2023-02-09 09:53:52,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 1a102fc8ca7411175545f4844f76a455, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:53:52,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:52,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1a102fc8ca7411175545f4844f76a455, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:53:52,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:52,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:52,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:52,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:52,706 INFO [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:52,707 DEBUG [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1 2023-02-09 09:53:52,707 DEBUG [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1 2023-02-09 09:53:52,707 INFO [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1a102fc8ca7411175545f4844f76a455 columnFamilyName family1 2023-02-09 09:53:52,714 DEBUG [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/9bfc8cb02c7e4662ae039f7c2166d052 2023-02-09 09:53:52,715 INFO [StoreOpener-1a102fc8ca7411175545f4844f76a455-1] regionserver.HStore(310): Store=1a102fc8ca7411175545f4844f76a455/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:52,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:52,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:52,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:53:52,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1a102fc8ca7411175545f4844f76a455; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66303601, jitterRate=-0.011999353766441345}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:53:52,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1a102fc8ca7411175545f4844f76a455: Running coprocessor pre-open hook at 1675936432704Writing region info on filesystem at 1675936432704Initializing all the Stores at 1675936432705 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675936432705Cleaning up temporary data from old regions at 1675936432718 (+13 ms)Cleaning up detritus from prior splits at 1675936432719 (+1 ms)Running coprocessor post-open hooks at 1675936432723 (+4 ms)Region opened successfully at 1675936432723 2023-02-09 09:53:52,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455., pid=17, masterSystemTime=1675936432700 2023-02-09 09:53:52,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:52,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:53:52,726 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1a102fc8ca7411175545f4844f76a455, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:52,726 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675936432726"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936432726"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936432726"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936432726"}]},"ts":"1675936432726"} 2023-02-09 09:53:52,730 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-09 09:53:52,730 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 1a102fc8ca7411175545f4844f76a455, server=jenkins-hbase4.apache.org,42719,1675936416061 in 181 msec 2023-02-09 09:53:52,733 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-02-09 09:53:52,733 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1a102fc8ca7411175545f4844f76a455, REOPEN/MOVE in 1.0400 sec 2023-02-09 09:53:52,736 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-09 09:53:52,736 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0570 sec 2023-02-09 09:53:52,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4890 sec 2023-02-09 09:53:52,740 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:52,740 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:52,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-02-09 09:53:52,742 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:52,743 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:52,747 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:52,747 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:52,748 INFO [Listener at localhost/42557] compactions.TestMobCompactor(278): alter status finished 2023-02-09 09:53:52,750 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 09:53:52,751 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 09:53:52,752 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 09:53:52,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:53:52,753 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937032753 2023-02-09 09:53:52,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:53:52,755 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-09 09:53:52,755 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 09:53:52,756 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x7fb4c5f1 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:53:52,868 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@10356889, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:53:52,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:52,869 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:52,869 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:52,869 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:52,869 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:53,703 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112859308421c9a94960a6899dd487754e16 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112859308421c9a94960a6899dd487754e16 2023-02-09 09:53:53,704 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 09:53:53,717 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:53:53,719 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43210, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:53:53,726 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:53,726 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:53,726 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:53:53,728 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42020, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:53:53,739 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3461339ce9394a1fba2100c9d93287c3 first=Optional[r05] last=Optional[r8] 2023-02-09 09:53:53,739 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=51 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3461339ce9394a1fba2100c9d93287c3}] 2023-02-09 09:53:53,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3461339ce9394a1fba2100c9d93287c3 for inclusion in 1a102fc8ca7411175545f4844f76a455/family1 2023-02-09 09:53:53,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-09 09:53:53,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:53:53,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 1a102fc8ca7411175545f4844f76a455: 2023-02-09 09:53:53,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3461339ce9394a1fba2100c9d93287c3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__bthf5om757ad8peil9vu84t8n2e5noh395mvaqtje2qjgfkbeev9ubom0l69sna4/family1/3461339ce9394a1fba2100c9d93287c3 2023-02-09 09:53:53,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__bthf5om757ad8peil9vu84t8n2e5noh395mvaqtje2qjgfkbeev9ubom0l69sna4/family1/3461339ce9394a1fba2100c9d93287c3 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/abf20bd9d16143598131440eeff12c24_SeqId_53_ 2023-02-09 09:53:53,754 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__bthf5om757ad8peil9vu84t8n2e5noh395mvaqtje2qjgfkbeev9ubom0l69sna4/family1/3461339ce9394a1fba2100c9d93287c3 into 1a102fc8ca7411175545f4844f76a455/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/abf20bd9d16143598131440eeff12c24_SeqId_53_ - updating store file list. 2023-02-09 09:53:53,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/abf20bd9d16143598131440eeff12c24_SeqId_53_ into 1a102fc8ca7411175545f4844f76a455/family1 2023-02-09 09:53:53,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__bthf5om757ad8peil9vu84t8n2e5noh395mvaqtje2qjgfkbeev9ubom0l69sna4/family1/3461339ce9394a1fba2100c9d93287c3 into 1a102fc8ca7411175545f4844f76a455/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/family1/abf20bd9d16143598131440eeff12c24_SeqId_53_) 2023-02-09 09:53:53,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__bthf5om757ad8peil9vu84t8n2e5noh395mvaqtje2qjgfkbeev9ubom0l69sna4/family1/3461339ce9394a1fba2100c9d93287c3 2023-02-09 09:53:53,763 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 09:53:53,764 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:53:53,766 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120e3958808be0345589948263cb6b0d6f4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120e3958808be0345589948263cb6b0d6f4 2023-02-09 09:53:53,767 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112814fca2afdba2430f97393f810b61a20c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112814fca2afdba2430f97393f810b61a20c 2023-02-09 09:53:53,767 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-02-09 09:53:53,767 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 09:53:53,768 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:53,768 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:53,768 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 09:53:53,768 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 09:53:53,768 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:53:53,769 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 09:53:53,771 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.0200 sec 2023-02-09 09:53:53,858 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=247 (was 210) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49042 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x358913c1-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64170@0x7fb4c5f1-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_118379199_15 at /127.0.0.1:49190 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:42719-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49164 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49208 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49282 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49212 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x358913c1-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64170@0x6fd61940 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1858355662.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64170@0x7fb4c5f1-SendThread(127.0.0.1:64170) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49092 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49246 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49014 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45feaf0d-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/cluster_355d5ed8-3bf0-2bc4-8778-241b2c013167/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45feaf0d-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64170@0x6fd61940-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49224 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49140 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49108 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49192 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x45feaf0d-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64170@0x7fb4c5f1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1858355662.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49180 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49064 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64170@0x6fd61940-SendThread(127.0.0.1:64170) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49214 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=590 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=271 (was 328), ProcessCount=168 (was 168), AvailableMemoryMB=4244 (was 4290) 2023-02-09 09:53:53,871 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=247, OpenFileDescriptor=590, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=168, AvailableMemoryMB=4243 2023-02-09 09:53:53,872 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:53,872 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:53,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 09:53:53,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:53,876 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:53:53,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-02-09 09:53:53,877 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:53,877 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:53,877 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:53:53,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 09:53:53,879 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf 2023-02-09 09:53:53,880 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf empty. 2023-02-09 09:53:53,880 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf 2023-02-09 09:53:53,880 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-02-09 09:53:53,978 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:53,978 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:53,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 09:53:54,180 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:54,180 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:54,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 09:53:54,290 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 09:53:54,291 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7a50216fadfaa2886b6c84c6392b99cf, NAME => 'testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:53:54,481 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:54,481 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:54,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 09:53:54,698 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:54,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 7a50216fadfaa2886b6c84c6392b99cf, disabling compactions & flushes 2023-02-09 09:53:54,699 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:54,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:54,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. after waiting 0 ms 2023-02-09 09:53:54,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:54,699 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:54,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 7a50216fadfaa2886b6c84c6392b99cf: Waiting for close lock at 1675936434698Disabling compacts and flushes for region at 1675936434698Disabling writes for close at 1675936434699 (+1 ms)Writing region close event to WAL at 1675936434699Closed at 1675936434699 2023-02-09 09:53:54,702 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:53:54,703 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675936434703"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936434703"}]},"ts":"1675936434703"} 2023-02-09 09:53:54,705 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 09:53:54,706 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:53:54,706 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936434706"}]},"ts":"1675936434706"} 2023-02-09 09:53:54,708 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-09 09:53:54,822 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7a50216fadfaa2886b6c84c6392b99cf, ASSIGN}] 2023-02-09 09:53:54,824 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7a50216fadfaa2886b6c84c6392b99cf, ASSIGN 2023-02-09 09:53:54,826 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7a50216fadfaa2886b6c84c6392b99cf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:53:54,977 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=7a50216fadfaa2886b6c84c6392b99cf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:54,977 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675936434977"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936434977"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936434977"}]},"ts":"1675936434977"} 2023-02-09 09:53:54,979 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 7a50216fadfaa2886b6c84c6392b99cf, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:53:54,983 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:54,983 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:54,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 09:53:55,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:55,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7a50216fadfaa2886b6c84c6392b99cf, NAME => 'testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:53:55,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 7a50216fadfaa2886b6c84c6392b99cf 2023-02-09 09:53:55,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:53:55,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7a50216fadfaa2886b6c84c6392b99cf 2023-02-09 09:53:55,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7a50216fadfaa2886b6c84c6392b99cf 2023-02-09 09:53:55,137 INFO [StoreOpener-7a50216fadfaa2886b6c84c6392b99cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7a50216fadfaa2886b6c84c6392b99cf 2023-02-09 09:53:55,139 DEBUG [StoreOpener-7a50216fadfaa2886b6c84c6392b99cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1 2023-02-09 09:53:55,139 DEBUG [StoreOpener-7a50216fadfaa2886b6c84c6392b99cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1 2023-02-09 09:53:55,139 INFO [StoreOpener-7a50216fadfaa2886b6c84c6392b99cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7a50216fadfaa2886b6c84c6392b99cf columnFamilyName family1 2023-02-09 09:53:55,141 INFO [StoreOpener-7a50216fadfaa2886b6c84c6392b99cf-1] regionserver.HStore(310): Store=7a50216fadfaa2886b6c84c6392b99cf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:53:55,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf 2023-02-09 09:53:55,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf 2023-02-09 09:53:55,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7a50216fadfaa2886b6c84c6392b99cf 2023-02-09 09:53:55,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:53:55,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7a50216fadfaa2886b6c84c6392b99cf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70397723, jitterRate=0.04900781810283661}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:53:55,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7a50216fadfaa2886b6c84c6392b99cf: Running coprocessor pre-open hook at 1675936435136Writing region info on filesystem at 1675936435136Initializing all the Stores at 1675936435137 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675936435137Cleaning up temporary data from old regions at 1675936435143 (+6 ms)Cleaning up detritus from prior splits at 1675936435144 (+1 ms)Running coprocessor post-open hooks at 1675936435149 (+5 ms)Region opened successfully at 1675936435149 2023-02-09 09:53:55,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf., pid=21, masterSystemTime=1675936435132 2023-02-09 09:53:55,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:55,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:55,152 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=7a50216fadfaa2886b6c84c6392b99cf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:55,153 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675936435152"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936435152"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936435152"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936435152"}]},"ts":"1675936435152"} 2023-02-09 09:53:55,156 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-02-09 09:53:55,156 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 7a50216fadfaa2886b6c84c6392b99cf, server=jenkins-hbase4.apache.org,42719,1675936416061 in 175 msec 2023-02-09 09:53:55,158 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-09 09:53:55,159 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7a50216fadfaa2886b6c84c6392b99cf, ASSIGN in 334 msec 2023-02-09 09:53:55,159 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:53:55,159 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936435159"}]},"ts":"1675936435159"} 2023-02-09 09:53:55,161 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-09 09:53:55,222 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:53:55,224 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.3490 sec 2023-02-09 09:53:55,598 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:55,984 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:55,984 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:55,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 09:53:55,986 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-02-09 09:53:56,020 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:56,020 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:56,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:56,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,023 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:56,023 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:56,024 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:53:56,024 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:53:56,024 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,024 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,026 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,026 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:56,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:56,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:56,026 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,026 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:56,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:56,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,027 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,027 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,027 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:53:56,027 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:56,028 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:53:56,028 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:56,028 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:56,028 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:56,028 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. started... 2023-02-09 09:53:56,029 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a50216fadfaa2886b6c84c6392b99cf 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 09:53:56,453 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130c556fe834cf3417d9c31d78a0a777f3c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130c556fe834cf3417d9c31d78a0a777f3c 2023-02-09 09:53:56,457 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/81004213b6084ad28331031a98fe0e03 2023-02-09 09:53:56,464 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/81004213b6084ad28331031a98fe0e03 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/81004213b6084ad28331031a98fe0e03 2023-02-09 09:53:56,470 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/81004213b6084ad28331031a98fe0e03, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-09 09:53:56,471 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 7a50216fadfaa2886b6c84c6392b99cf in 442ms, sequenceid=5, compaction requested=false 2023-02-09 09:53:56,472 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:53:56,472 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:56,472 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:56,472 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:56,472 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,472 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:53:56,472 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:53:56,474 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,474 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:56,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:56,474 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,474 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:56,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:56,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:56,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:56,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:53:56,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@484d28f7[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:56,476 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:53:56,476 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,477 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,478 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:53:56,478 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:53:56,478 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,478 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:56,478 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:56,480 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:53:56,480 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:56,480 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,480 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:53:56,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:56,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:56,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:56,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:56,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:56,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:53:56,484 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:53:56,484 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:56,485 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:56,485 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,485 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:56,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:56,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:56,487 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:56,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:56,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:56,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:56,487 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:56,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,489 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,492 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,492 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,492 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:56,492 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:56,492 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,492 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:56,492 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:56,492 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,492 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:56,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:56,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:53:56,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:56,492 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,493 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,493 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:56,493 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:56,493 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:56,593 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:56,593 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:56,594 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:56,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:56,600 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:56,600 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:56,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:56,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:56,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,603 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:56,603 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:56,603 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:53:56,603 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:53:56,604 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,604 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,605 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,605 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:56,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:56,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:56,606 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:56,606 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:56,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:56,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,607 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:53:56,607 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:56,607 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:53:56,607 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:56,608 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:53:56,608 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:56,608 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:56,608 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:56,608 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. started... 2023-02-09 09:53:56,608 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a50216fadfaa2886b6c84c6392b99cf 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 09:53:57,026 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112848e7d9a6024042a8bfc5ca04082938fa to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112848e7d9a6024042a8bfc5ca04082938fa 2023-02-09 09:53:57,433 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/2fe7b7447c1e42d5bcca48318ff26520 2023-02-09 09:53:57,441 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/2fe7b7447c1e42d5bcca48318ff26520 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/2fe7b7447c1e42d5bcca48318ff26520 2023-02-09 09:53:57,447 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/2fe7b7447c1e42d5bcca48318ff26520, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 09:53:57,449 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 7a50216fadfaa2886b6c84c6392b99cf in 841ms, sequenceid=9, compaction requested=false 2023-02-09 09:53:57,449 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:53:57,449 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:57,449 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:57,449 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:57,449 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,449 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:53:57,449 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:53:57,452 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,452 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:57,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:57,453 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,453 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:57,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:57,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:57,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:57,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:53:57,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@556e944f[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:57,455 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:53:57,455 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,457 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,458 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:53:57,458 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:53:57,458 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:57,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:57,460 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:53:57,460 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,460 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:57,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:57,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:57,460 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:53:57,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:57,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:57,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:57,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:57,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:53:57,464 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:53:57,464 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:57,464 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:57,464 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,464 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:57,466 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:57,466 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,466 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:57,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:57,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:57,467 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:57,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,469 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:57,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,482 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,482 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,482 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:57,482 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:53:57,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:57,482 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:57,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:57,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:57,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:57,482 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:57,482 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,482 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,483 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,483 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,483 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,483 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:57,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:57,483 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:57,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:57,584 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:57,584 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:57,584 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:57,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:57,590 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:57,590 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:57,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:57,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:57,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,593 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:57,593 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:57,594 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:53:57,594 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:53:57,594 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,594 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,596 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,596 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:57,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:57,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:57,596 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:57,596 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:57,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:57,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,597 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,597 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:53:57,597 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:57,597 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:53:57,597 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:57,598 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:53:57,598 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:57,598 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:57,598 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:57,598 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. started... 2023-02-09 09:53:57,599 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a50216fadfaa2886b6c84c6392b99cf 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:57,921 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:53:58,015 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201faaac5ec1912430aadb2982aa5110939 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201faaac5ec1912430aadb2982aa5110939 2023-02-09 09:53:58,423 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/611fe245dc1942a4bc20ca0af468ebc3 2023-02-09 09:53:58,431 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/611fe245dc1942a4bc20ca0af468ebc3 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/611fe245dc1942a4bc20ca0af468ebc3 2023-02-09 09:53:58,438 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/611fe245dc1942a4bc20ca0af468ebc3, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 09:53:58,439 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7a50216fadfaa2886b6c84c6392b99cf in 841ms, sequenceid=13, compaction requested=true 2023-02-09 09:53:58,439 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:53:58,439 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:53:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:53:58,441 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,441 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:58,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:58,442 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,442 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:58,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:58,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:58,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:58,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:53:58,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74673946[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:58,443 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:53:58,444 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,445 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,445 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:53:58,445 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:53:58,445 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,445 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:58,445 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:53:58,448 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:58,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:53:58,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:58,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:58,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:58,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:58,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:58,452 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:53:58,452 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:53:58,452 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:58,452 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:58,452 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,452 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:58,454 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,454 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:58,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:58,454 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:58,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:58,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:58,455 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:58,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:58,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,457 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:58,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,461 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,461 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,461 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,461 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:58,461 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:58,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:58,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:58,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:58,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:58,462 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,462 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:53:58,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:58,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:58,462 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:58,463 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,463 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,464 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:58,464 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:58,464 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,564 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:58,564 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:58,564 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:58,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:58,570 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:58,570 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:58,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:58,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:58,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,573 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:58,573 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:58,573 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:53:58,573 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:53:58,574 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,574 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,575 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:58,575 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:58,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:58,576 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:58,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,576 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:58,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:58,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,577 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:53:58,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:58,577 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:53:58,577 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:58,578 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:53:58,578 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:58,578 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:58,578 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. started... 2023-02-09 09:53:58,578 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:58,578 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a50216fadfaa2886b6c84c6392b99cf 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:58,996 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205306b00abffec476d87da985768b84d89 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205306b00abffec476d87da985768b84d89 2023-02-09 09:53:59,410 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/7550d45824cd4a1dac701371b29f4b1e 2023-02-09 09:53:59,418 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/7550d45824cd4a1dac701371b29f4b1e as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/7550d45824cd4a1dac701371b29f4b1e 2023-02-09 09:53:59,424 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/7550d45824cd4a1dac701371b29f4b1e, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 09:53:59,433 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7a50216fadfaa2886b6c84c6392b99cf in 855ms, sequenceid=17, compaction requested=true 2023-02-09 09:53:59,433 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:53:59,433 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:59,433 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:59,433 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:59,433 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,433 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:53:59,433 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:53:59,436 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,436 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:59,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:59,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:59,436 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,436 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:59,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:59,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:59,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:53:59,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27b78bcc[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:59,438 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:53:59,438 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,439 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,439 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:53:59,439 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:53:59,439 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:59,439 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,439 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:59,441 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,441 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:53:59,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:59,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:59,441 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:59,441 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:53:59,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:59,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:59,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:59,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:59,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:53:59,444 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:53:59,444 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:59,444 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:59,444 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,444 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:59,446 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,446 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:59,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:59,446 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:59,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,447 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:59,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:59,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:59,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:59,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,448 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,451 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,452 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,452 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,452 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,452 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:59,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:53:59,452 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:59,452 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:59,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:59,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:59,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:59,453 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,453 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,453 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:59,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:59,453 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:59,453 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:59,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:59,553 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:59,554 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:59,554 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:59,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:59,560 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:59,560 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:59,561 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:59,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:59,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,564 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:59,564 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:59,565 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:53:59,565 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:53:59,565 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,565 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,567 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,567 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:59,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:59,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:59,567 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,567 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:59,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,568 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:59,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,568 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:53:59,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,569 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:53:59,569 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:59,570 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:53:59,570 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:59,570 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:59,570 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:59,570 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. started... 2023-02-09 09:53:59,570 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a50216fadfaa2886b6c84c6392b99cf 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:53:59,594 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228895ef8572e2946ccb1dde2359dd5ab1c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228895ef8572e2946ccb1dde2359dd5ab1c 2023-02-09 09:53:59,606 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/77c6bf22f9894434859568cec373b1d7 2023-02-09 09:53:59,614 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/77c6bf22f9894434859568cec373b1d7 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/77c6bf22f9894434859568cec373b1d7 2023-02-09 09:53:59,622 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/77c6bf22f9894434859568cec373b1d7, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 09:53:59,624 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7a50216fadfaa2886b6c84c6392b99cf in 54ms, sequenceid=21, compaction requested=true 2023-02-09 09:53:59,624 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:53:59,624 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:59,624 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:53:59,624 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:53:59,624 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,624 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:53:59,624 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:53:59,626 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,626 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:59,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:59,627 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,627 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:53:59,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:59,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:59,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:59,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:53:59,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61c155d[Count = 0] remaining members to acquire global barrier 2023-02-09 09:53:59,635 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:53:59,635 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,636 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,637 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:53:59,637 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:53:59,637 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:53:59,637 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,637 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:53:59,640 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,640 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:53:59,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:59,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:59,640 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:59,640 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:53:59,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:59,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:59,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:59,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:53:59,644 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:53:59,644 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:53:59,644 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:53:59,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:53:59,644 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,644 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:53:59,651 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,651 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,651 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:53:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:53:59,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:59,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:53:59,654 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:53:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,655 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:53:59,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,659 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:59,659 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:53:59,659 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,660 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:53:59,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:59,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:53:59,664 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:59,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:53:59,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:59,664 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:53:59,664 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:53:59,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:53:59,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:53:59,765 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:53:59,765 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:59,765 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:59,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:53:59,772 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:53:59,772 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:53:59,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:53:59,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:53:59,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,775 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:53:59,775 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:59,776 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:53:59,776 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:53:59,777 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,777 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,779 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:53:59,779 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:53:59,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:53:59,779 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:53:59,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,780 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:53:59,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:53:59,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,780 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,781 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:53:59,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:53:59,781 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:53:59,781 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:53:59,781 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:53:59,782 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:53:59,782 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:53:59,782 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:53:59,782 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. started... 2023-02-09 09:53:59,782 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a50216fadfaa2886b6c84c6392b99cf 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:00,213 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d0d2ee14121c4caaa84aa27e7ad7c11e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d0d2ee14121c4caaa84aa27e7ad7c11e 2023-02-09 09:54:00,619 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/82e070a6fc9d44baae1eed1aae429ecd 2023-02-09 09:54:00,627 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/82e070a6fc9d44baae1eed1aae429ecd as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/82e070a6fc9d44baae1eed1aae429ecd 2023-02-09 09:54:00,634 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/82e070a6fc9d44baae1eed1aae429ecd, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 09:54:00,635 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7a50216fadfaa2886b6c84c6392b99cf in 853ms, sequenceid=25, compaction requested=true 2023-02-09 09:54:00,635 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:54:00,635 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:00,635 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:00,635 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:00,635 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,635 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:54:00,636 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:54:00,637 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,637 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:00,638 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,638 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:00,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:00,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:54:00,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2771253d[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:00,639 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:00,640 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,641 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,641 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:00,641 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:54:00,641 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,641 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:00,641 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:00,643 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:54:00,643 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,643 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,643 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:54:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:00,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:00,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:00,647 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:54:00,647 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:00,647 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:00,647 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,647 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:00,654 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,654 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:00,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,654 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:00,655 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:00,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,658 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:00,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:54:00,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:00,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:00,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:00,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:00,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,660 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:00,660 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,660 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:00,661 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:00,661 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:00,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:00,661 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,761 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:00,762 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:00,762 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:00,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:00,767 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:00,767 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:00,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:00,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:00,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,770 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:54:00,770 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:00,770 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:54:00,770 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:00,771 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,771 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,772 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:00,772 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:00,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:00,772 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:00,772 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:00,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,773 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:00,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,773 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,773 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:54:00,773 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:00,773 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:54:00,773 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:00,774 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:54:00,774 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:00,774 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:00,774 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:00,774 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. started... 2023-02-09 09:54:00,774 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a50216fadfaa2886b6c84c6392b99cf 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:01,193 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101538737f3a6224a21a98b1ef3c8441c7d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101538737f3a6224a21a98b1ef3c8441c7d 2023-02-09 09:54:01,601 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/4a30a3afd55a476c82ca92ad4cbf0071 2023-02-09 09:54:01,608 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/4a30a3afd55a476c82ca92ad4cbf0071 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/4a30a3afd55a476c82ca92ad4cbf0071 2023-02-09 09:54:01,614 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/4a30a3afd55a476c82ca92ad4cbf0071, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 09:54:01,615 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7a50216fadfaa2886b6c84c6392b99cf in 841ms, sequenceid=29, compaction requested=true 2023-02-09 09:54:01,615 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:54:01,615 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:54:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:54:01,617 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,617 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:01,617 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,617 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:01,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:01,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:01,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:54:01,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a11859c[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:01,619 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:01,619 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,620 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,620 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,621 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:01,621 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:01,621 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:54:01,621 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:01,622 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,622 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:54:01,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:01,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:01,622 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:01,622 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:54:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:01,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:01,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:01,625 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:54:01,625 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:01,626 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:01,626 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,626 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:01,629 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,629 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:01,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:01,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,629 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:01,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:01,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:01,630 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:01,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:01,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,631 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:01,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,634 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,634 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:01,634 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,634 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:01,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:54:01,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:01,634 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,634 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:01,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:01,635 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:54:01,635 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,635 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:01,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:01,635 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:01,635 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,635 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:01,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:01,735 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:01,735 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:01,736 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:01,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:01,743 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:01,743 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:01,744 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:01,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:01,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,746 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:54:01,746 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:01,746 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:54:01,746 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:01,747 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,747 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,748 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,748 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:01,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:01,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:01,748 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:01,748 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:01,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:01,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,750 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:54:01,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:01,750 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:54:01,750 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:01,750 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:54:01,751 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:01,751 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:01,751 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:01,751 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. started... 2023-02-09 09:54:01,751 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a50216fadfaa2886b6c84c6392b99cf 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:02,168 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030b788ad9837b42759ed20b068d3e65bd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601030b788ad9837b42759ed20b068d3e65bd 2023-02-09 09:54:02,174 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/12a59f2024cd47949963ec9fbf6422a6 2023-02-09 09:54:02,181 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/12a59f2024cd47949963ec9fbf6422a6 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/12a59f2024cd47949963ec9fbf6422a6 2023-02-09 09:54:02,186 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/12a59f2024cd47949963ec9fbf6422a6, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 09:54:02,187 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7a50216fadfaa2886b6c84c6392b99cf in 436ms, sequenceid=33, compaction requested=true 2023-02-09 09:54:02,188 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:54:02,188 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:02,188 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:02,188 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:02,188 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,188 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:54:02,188 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:54:02,190 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,190 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:02,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:02,190 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:02,191 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:02,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:02,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:02,192 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:54:02,192 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71639af9[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:02,192 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:02,192 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,195 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,195 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:02,195 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:54:02,195 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,195 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:02,195 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:02,200 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:54:02,200 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,200 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:02,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:02,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:02,201 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:54:02,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:02,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:02,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:02,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:02,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:02,204 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:54:02,204 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:02,204 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:02,204 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,204 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:02,206 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,206 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:02,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,206 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:02,207 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:02,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:02,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,208 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:02,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,212 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,212 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:02,212 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,212 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:02,212 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:54:02,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:02,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:02,212 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:02,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:02,212 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,213 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,213 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:02,213 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:02,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:02,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:02,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:54:02,313 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:02,314 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:02,314 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:02,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:02,319 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:02,319 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:02,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:02,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:02,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,323 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:54:02,323 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:02,324 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:54:02,324 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:02,324 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,324 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,325 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,325 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:02,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:02,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:02,326 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,326 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:02,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:02,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:54:02,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,327 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:54:02,328 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:02,328 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:54:02,328 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:02,328 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:02,328 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:02,328 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. started... 2023-02-09 09:54:02,328 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a50216fadfaa2886b6c84c6392b99cf 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:02,349 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907818ce046ac1a48ecbab0cae9f32a8f3b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907818ce046ac1a48ecbab0cae9f32a8f3b 2023-02-09 09:54:02,755 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/5292e92dbbe143bf8a688f66cbb190b7 2023-02-09 09:54:02,762 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/5292e92dbbe143bf8a688f66cbb190b7 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/5292e92dbbe143bf8a688f66cbb190b7 2023-02-09 09:54:02,769 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/5292e92dbbe143bf8a688f66cbb190b7, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 09:54:02,770 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7a50216fadfaa2886b6c84c6392b99cf in 442ms, sequenceid=37, compaction requested=true 2023-02-09 09:54:02,770 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:54:02,770 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:02,771 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:02,771 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:02,771 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,771 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:54:02,771 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:54:02,772 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,772 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:02,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:02,773 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,773 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:02,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:02,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:02,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:02,775 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:54:02,775 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17b3df75[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:02,775 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:02,775 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,776 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:02,777 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,777 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:02,777 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:54:02,777 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:02,779 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,779 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:54:02,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:02,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:02,779 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:02,779 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:54:02,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:02,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:02,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:02,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:02,782 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:54:02,782 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:02,782 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:02,782 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,782 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:02,784 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,784 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:02,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,785 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:02,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:02,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:02,787 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,788 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,790 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,790 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:02,790 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,791 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:02,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:02,791 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:54:02,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:02,791 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:02,791 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:02,791 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,792 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,792 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,792 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:02,792 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:02,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:02,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:02,892 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:02,892 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:02,892 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:02,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:02,898 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:02,898 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:02,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:02,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:02,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,901 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:54:02,901 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:02,902 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:54:02,902 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:02,902 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,902 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,904 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:02,904 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:02,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:02,904 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:02,904 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:02,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,905 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:02,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,905 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,905 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:54:02,905 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:02,905 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:54:02,905 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:02,907 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:54:02,907 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:02,907 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:02,907 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:02,907 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. started... 2023-02-09 09:54:02,907 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a50216fadfaa2886b6c84c6392b99cf 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:03,326 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112005a66f5e66704c96aec7aec24cdf2b11 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112005a66f5e66704c96aec7aec24cdf2b11 2023-02-09 09:54:03,332 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/f882b002abca480ab3f07bf6d4d8e74f 2023-02-09 09:54:03,339 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/f882b002abca480ab3f07bf6d4d8e74f as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/f882b002abca480ab3f07bf6d4d8e74f 2023-02-09 09:54:03,345 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/f882b002abca480ab3f07bf6d4d8e74f, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 09:54:03,346 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7a50216fadfaa2886b6c84c6392b99cf in 439ms, sequenceid=41, compaction requested=true 2023-02-09 09:54:03,347 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:54:03,347 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:03,347 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:03,347 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:03,347 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,347 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:54:03,347 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:54:03,350 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,350 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:03,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:03,350 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:03,350 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:03,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:03,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:03,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:54:03,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39454873[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:03,352 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:03,352 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,354 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,354 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:03,354 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:54:03,354 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:03,354 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,354 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:03,355 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,355 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:54:03,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:03,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:03,355 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:03,355 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:54:03,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:03,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:03,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:03,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:03,359 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:03,359 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:54:03,359 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:03,359 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:03,359 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,359 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:03,360 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,361 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:03,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,361 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:03,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:03,361 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:03,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:03,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:03,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,362 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:03,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,366 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:03,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:03,366 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:03,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:54:03,366 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,366 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 09:54:03,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:03,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:03,366 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:03,367 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,368 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,368 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:03,368 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:03,368 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:03,368 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:03,368 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:03,368 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,468 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:03,468 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:03,468 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:03,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:03,471 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-02-09 09:54:03,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:54:03,474 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:54:03,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:03,475 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937043475 2023-02-09 09:54:03,476 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:54:03,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-09 09:54:03,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:54:03,478 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x68a0246a to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:54:03,490 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@536a68d7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:54:03,490 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:54:03,544 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ec138342e70b4e5ab39c78ec20986542 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103ec138342e70b4e5ab39c78ec20986542 2023-02-09 09:54:03,546 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:54:03,562 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:54:03,563 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60812, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:54:03,573 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:03,573 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:03,574 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:54:03,575 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35778, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:54:03,591 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/54fc2b6ce1e944d48b583d7e6dd0a5cd first=Optional[r5] last=Optional[r6] 2023-02-09 09:54:03,591 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/54fc2b6ce1e944d48b583d7e6dd0a5cd}] 2023-02-09 09:54:03,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/54fc2b6ce1e944d48b583d7e6dd0a5cd for inclusion in 7a50216fadfaa2886b6c84c6392b99cf/family1 2023-02-09 09:54:03,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-09 09:54:03,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:54:03,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:54:03,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/54fc2b6ce1e944d48b583d7e6dd0a5cd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__i2h2hosmpgb0bujkdib7017vodbhinb2g8b3g1hi7v43lo2n3oe5pfiuh02lqd90/family1/54fc2b6ce1e944d48b583d7e6dd0a5cd 2023-02-09 09:54:03,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__i2h2hosmpgb0bujkdib7017vodbhinb2g8b3g1hi7v43lo2n3oe5pfiuh02lqd90/family1/54fc2b6ce1e944d48b583d7e6dd0a5cd as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/b1e4add6b5a9412d830c6952fc815f04_SeqId_44_ 2023-02-09 09:54:03,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__i2h2hosmpgb0bujkdib7017vodbhinb2g8b3g1hi7v43lo2n3oe5pfiuh02lqd90/family1/54fc2b6ce1e944d48b583d7e6dd0a5cd into 7a50216fadfaa2886b6c84c6392b99cf/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/b1e4add6b5a9412d830c6952fc815f04_SeqId_44_ - updating store file list. 2023-02-09 09:54:03,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/b1e4add6b5a9412d830c6952fc815f04_SeqId_44_ into 7a50216fadfaa2886b6c84c6392b99cf/family1 2023-02-09 09:54:03,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__i2h2hosmpgb0bujkdib7017vodbhinb2g8b3g1hi7v43lo2n3oe5pfiuh02lqd90/family1/54fc2b6ce1e944d48b583d7e6dd0a5cd into 7a50216fadfaa2886b6c84c6392b99cf/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/b1e4add6b5a9412d830c6952fc815f04_SeqId_44_) 2023-02-09 09:54:03,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__i2h2hosmpgb0bujkdib7017vodbhinb2g8b3g1hi7v43lo2n3oe5pfiuh02lqd90/family1/54fc2b6ce1e944d48b583d7e6dd0a5cd 2023-02-09 09:54:03,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 09:54:03,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 09:54:03,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): 7a50216fadfaa2886b6c84c6392b99cf/family1 is initiating minor compaction 2023-02-09 09:54:03,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1f1e9c0d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:03,623 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7a50216fadfaa2886b6c84c6392b99cf/family1 in testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:03,623 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/2fe7b7447c1e42d5bcca48318ff26520, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/611fe245dc1942a4bc20ca0af468ebc3, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/7550d45824cd4a1dac701371b29f4b1e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/77c6bf22f9894434859568cec373b1d7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/82e070a6fc9d44baae1eed1aae429ecd, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/4a30a3afd55a476c82ca92ad4cbf0071, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/12a59f2024cd47949963ec9fbf6422a6, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/5292e92dbbe143bf8a688f66cbb190b7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/f882b002abca480ab3f07bf6d4d8e74f, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/b1e4add6b5a9412d830c6952fc815f04_SeqId_44_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp, totalSize=49.7 K 2023-02-09 09:54:03,623 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 2fe7b7447c1e42d5bcca48318ff26520, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 09:54:03,623 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 611fe245dc1942a4bc20ca0af468ebc3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 09:54:03,623 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 7550d45824cd4a1dac701371b29f4b1e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 09:54:03,623 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 77c6bf22f9894434859568cec373b1d7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 09:54:03,623 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 82e070a6fc9d44baae1eed1aae429ecd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 09:54:03,624 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 4a30a3afd55a476c82ca92ad4cbf0071, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 09:54:03,624 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 12a59f2024cd47949963ec9fbf6422a6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 09:54:03,624 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 5292e92dbbe143bf8a688f66cbb190b7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 09:54:03,624 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting f882b002abca480ab3f07bf6d4d8e74f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 09:54:03,624 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting b1e4add6b5a9412d830c6952fc815f04_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 09:54:03,628 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:54:03,630 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:03,632 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101538737f3a6224a21a98b1ef3c8441c7d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101538737f3a6224a21a98b1ef3c8441c7d 2023-02-09 09:54:03,634 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601030b788ad9837b42759ed20b068d3e65bd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601030b788ad9837b42759ed20b068d3e65bd 2023-02-09 09:54:03,634 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-09 09:54:03,634 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-09 09:54:03,635 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:54:03,635 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:54:03,684 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:54:03,684 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7fb4c5f1 to 127.0.0.1:64170 2023-02-09 09:54:03,684 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:54:03,697 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:54:03,697 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6fd61940 to 127.0.0.1:64170 2023-02-09 09:54:03,697 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:54:03,761 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7a50216fadfaa2886b6c84c6392b99cf#family1#compaction#22 average throughput is 0.27 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:54:03,789 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/acf098c90498451d8e71769f25dbc9e1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/acf098c90498451d8e71769f25dbc9e1 2023-02-09 09:54:03,797 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 7a50216fadfaa2886b6c84c6392b99cf/family1 of 7a50216fadfaa2886b6c84c6392b99cf into acf098c90498451d8e71769f25dbc9e1(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:54:03,797 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:54:03,798 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1675936443622; duration=0sec 2023-02-09 09:54:03,798 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:03,846 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/2fe7b7447c1e42d5bcca48318ff26520, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/611fe245dc1942a4bc20ca0af468ebc3, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/7550d45824cd4a1dac701371b29f4b1e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/77c6bf22f9894434859568cec373b1d7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/82e070a6fc9d44baae1eed1aae429ecd, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/4a30a3afd55a476c82ca92ad4cbf0071, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/12a59f2024cd47949963ec9fbf6422a6, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/5292e92dbbe143bf8a688f66cbb190b7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/f882b002abca480ab3f07bf6d4d8e74f, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/b1e4add6b5a9412d830c6952fc815f04_SeqId_44_] to archive 2023-02-09 09:54:03,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:03,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/2fe7b7447c1e42d5bcca48318ff26520 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/2fe7b7447c1e42d5bcca48318ff26520 2023-02-09 09:54:03,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/611fe245dc1942a4bc20ca0af468ebc3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/611fe245dc1942a4bc20ca0af468ebc3 2023-02-09 09:54:03,853 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/7550d45824cd4a1dac701371b29f4b1e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/7550d45824cd4a1dac701371b29f4b1e 2023-02-09 09:54:03,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/77c6bf22f9894434859568cec373b1d7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/77c6bf22f9894434859568cec373b1d7 2023-02-09 09:54:03,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/82e070a6fc9d44baae1eed1aae429ecd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/82e070a6fc9d44baae1eed1aae429ecd 2023-02-09 09:54:03,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/4a30a3afd55a476c82ca92ad4cbf0071 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/4a30a3afd55a476c82ca92ad4cbf0071 2023-02-09 09:54:03,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/12a59f2024cd47949963ec9fbf6422a6 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/12a59f2024cd47949963ec9fbf6422a6 2023-02-09 09:54:03,861 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/5292e92dbbe143bf8a688f66cbb190b7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/5292e92dbbe143bf8a688f66cbb190b7 2023-02-09 09:54:03,862 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/f882b002abca480ab3f07bf6d4d8e74f to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/f882b002abca480ab3f07bf6d4d8e74f 2023-02-09 09:54:03,864 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/b1e4add6b5a9412d830c6952fc815f04_SeqId_44_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/b1e4add6b5a9412d830c6952fc815f04_SeqId_44_ 2023-02-09 09:54:04,591 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231fa03ff276716434fa208d6ce945c5e64 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231fa03ff276716434fa208d6ce945c5e64 2023-02-09 09:54:04,592 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:54:04,610 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:04,610 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:04,621 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/144efeebfe6c4fbc8e3c3f3b56748c09 first=Optional[r1] last=Optional[r4] 2023-02-09 09:54:04,622 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/144efeebfe6c4fbc8e3c3f3b56748c09}] 2023-02-09 09:54:04,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/144efeebfe6c4fbc8e3c3f3b56748c09 for inclusion in 7a50216fadfaa2886b6c84c6392b99cf/family1 2023-02-09 09:54:04,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-09 09:54:04,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:54:04,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:54:04,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/144efeebfe6c4fbc8e3c3f3b56748c09 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vg052s3mnbuuf0dhhihabhvobpv3sbu9v6di1311u2ie8t7tnv7hjilr3hr5q6th/family1/144efeebfe6c4fbc8e3c3f3b56748c09 2023-02-09 09:54:04,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vg052s3mnbuuf0dhhihabhvobpv3sbu9v6di1311u2ie8t7tnv7hjilr3hr5q6th/family1/144efeebfe6c4fbc8e3c3f3b56748c09 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/cf926118c649478992d0089bfc5ecc2a_SeqId_47_ 2023-02-09 09:54:04,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vg052s3mnbuuf0dhhihabhvobpv3sbu9v6di1311u2ie8t7tnv7hjilr3hr5q6th/family1/144efeebfe6c4fbc8e3c3f3b56748c09 into 7a50216fadfaa2886b6c84c6392b99cf/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/cf926118c649478992d0089bfc5ecc2a_SeqId_47_ - updating store file list. 2023-02-09 09:54:04,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/cf926118c649478992d0089bfc5ecc2a_SeqId_47_ into 7a50216fadfaa2886b6c84c6392b99cf/family1 2023-02-09 09:54:04,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vg052s3mnbuuf0dhhihabhvobpv3sbu9v6di1311u2ie8t7tnv7hjilr3hr5q6th/family1/144efeebfe6c4fbc8e3c3f3b56748c09 into 7a50216fadfaa2886b6c84c6392b99cf/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/cf926118c649478992d0089bfc5ecc2a_SeqId_47_) 2023-02-09 09:54:04,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vg052s3mnbuuf0dhhihabhvobpv3sbu9v6di1311u2ie8t7tnv7hjilr3hr5q6th/family1/144efeebfe6c4fbc8e3c3f3b56748c09 2023-02-09 09:54:04,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 09:54:04,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 09:54:04,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): 7a50216fadfaa2886b6c84c6392b99cf/family1 is initiating minor compaction (all files) 2023-02-09 09:54:04,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@310d6b4a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:04,641 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7a50216fadfaa2886b6c84c6392b99cf/family1 in testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:54:04,641 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/81004213b6084ad28331031a98fe0e03, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/acf098c90498451d8e71769f25dbc9e1, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/cf926118c649478992d0089bfc5ecc2a_SeqId_47_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp, totalSize=178.1 K 2023-02-09 09:54:04,642 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 81004213b6084ad28331031a98fe0e03, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840054 2023-02-09 09:54:04,642 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting acf098c90498451d8e71769f25dbc9e1, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240054 2023-02-09 09:54:04,643 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting cf926118c649478992d0089bfc5ecc2a_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965240054 2023-02-09 09:54:04,643 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:54:04,644 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:04,646 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201faaac5ec1912430aadb2982aa5110939 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201faaac5ec1912430aadb2982aa5110939 2023-02-09 09:54:04,647 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205306b00abffec476d87da985768b84d89 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205306b00abffec476d87da985768b84d89 2023-02-09 09:54:04,649 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228895ef8572e2946ccb1dde2359dd5ab1c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228895ef8572e2946ccb1dde2359dd5ab1c 2023-02-09 09:54:04,651 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d0d2ee14121c4caaa84aa27e7ad7c11e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d0d2ee14121c4caaa84aa27e7ad7c11e 2023-02-09 09:54:04,651 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-09 09:54:04,652 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-09 09:54:04,652 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:54:04,652 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:54:04,670 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7a50216fadfaa2886b6c84c6392b99cf#family1#compaction#23 average throughput is 9.73 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:54:04,693 WARN [DataStreamer for file /user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123156439e8a9c5c4d719cab49379463b71c_del block BP-294638041-172.31.14.131-1675936411864:blk_1073741899_1075] hdfs.DataStreamer(823): DataStreamer Exception java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:477) at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at java.io.DataOutputStream.flush(DataOutputStream.java:123) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:772) 2023-02-09 09:54:04,732 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/.tmp/family1/bec6ad0e5ac04cbc97c4ab8510120991 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/bec6ad0e5ac04cbc97c4ab8510120991 2023-02-09 09:54:04,740 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 7a50216fadfaa2886b6c84c6392b99cf/family1 of 7a50216fadfaa2886b6c84c6392b99cf into bec6ad0e5ac04cbc97c4ab8510120991(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:54:04,740 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:54:04,740 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1675936444641; duration=0sec 2023-02-09 09:54:04,740 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:04,829 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/81004213b6084ad28331031a98fe0e03, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/acf098c90498451d8e71769f25dbc9e1, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/cf926118c649478992d0089bfc5ecc2a_SeqId_47_] to archive 2023-02-09 09:54:04,830 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:04,833 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/81004213b6084ad28331031a98fe0e03 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/81004213b6084ad28331031a98fe0e03 2023-02-09 09:54:04,834 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/acf098c90498451d8e71769f25dbc9e1 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/acf098c90498451d8e71769f25dbc9e1 2023-02-09 09:54:04,836 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/cf926118c649478992d0089bfc5ecc2a_SeqId_47_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/cf926118c649478992d0089bfc5ecc2a_SeqId_47_ 2023-02-09 09:54:05,121 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130c77279cc71994263802140619153d2ae to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130c77279cc71994263802140619153d2ae 2023-02-09 09:54:05,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:54:05,139 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:05,139 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:05,148 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/73546f335d45433ab6714907279de7c4 first=Optional[r00] last=Optional[r8] 2023-02-09 09:54:05,149 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/73546f335d45433ab6714907279de7c4}] 2023-02-09 09:54:05,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/73546f335d45433ab6714907279de7c4 for inclusion in 7a50216fadfaa2886b6c84c6392b99cf/family1 2023-02-09 09:54:05,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-09 09:54:05,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:54:05,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 7a50216fadfaa2886b6c84c6392b99cf: 2023-02-09 09:54:05,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/73546f335d45433ab6714907279de7c4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__f2p5tj40e9l2mntaqhi89it9akr8fi4rna7ccdbfekt946hia4hf23121jvqbv65/family1/73546f335d45433ab6714907279de7c4 2023-02-09 09:54:05,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__f2p5tj40e9l2mntaqhi89it9akr8fi4rna7ccdbfekt946hia4hf23121jvqbv65/family1/73546f335d45433ab6714907279de7c4 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/475e974f207f4fc2a12b1f51ea74645d_SeqId_50_ 2023-02-09 09:54:05,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__f2p5tj40e9l2mntaqhi89it9akr8fi4rna7ccdbfekt946hia4hf23121jvqbv65/family1/73546f335d45433ab6714907279de7c4 into 7a50216fadfaa2886b6c84c6392b99cf/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/475e974f207f4fc2a12b1f51ea74645d_SeqId_50_ - updating store file list. 2023-02-09 09:54:05,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/475e974f207f4fc2a12b1f51ea74645d_SeqId_50_ into 7a50216fadfaa2886b6c84c6392b99cf/family1 2023-02-09 09:54:05,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__f2p5tj40e9l2mntaqhi89it9akr8fi4rna7ccdbfekt946hia4hf23121jvqbv65/family1/73546f335d45433ab6714907279de7c4 into 7a50216fadfaa2886b6c84c6392b99cf/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/family1/475e974f207f4fc2a12b1f51ea74645d_SeqId_50_) 2023-02-09 09:54:05,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithMonthlyPolicy__f2p5tj40e9l2mntaqhi89it9akr8fi4rna7ccdbfekt946hia4hf23121jvqbv65/family1/73546f335d45433ab6714907279de7c4 2023-02-09 09:54:05,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:54:05,166 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:05,167 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112005a66f5e66704c96aec7aec24cdf2b11 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112005a66f5e66704c96aec7aec24cdf2b11 2023-02-09 09:54:05,168 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112848e7d9a6024042a8bfc5ca04082938fa to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112848e7d9a6024042a8bfc5ca04082938fa 2023-02-09 09:54:05,169 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130c556fe834cf3417d9c31d78a0a777f3c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130c556fe834cf3417d9c31d78a0a777f3c 2023-02-09 09:54:05,169 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-09 09:54:05,169 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 09:54:05,170 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:54:05,170 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:54:05,170 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:54:05,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:54:05,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:05,171 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:54:05,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.6990 sec 2023-02-09 09:54:05,284 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=250 (was 247) Potentially hanging thread: hconnection-0x737c9c8-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:46050 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:56912 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x635c2370-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:57006 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x635c2370-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x635c2370-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:46076 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x635c2370-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:56948 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x737c9c8-shared-pool-22 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:57118 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:57046 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64170@0x68a0246a sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1858355662.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:57042 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:56918 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x635c2370-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/cluster_355d5ed8-3bf0-2bc4-8778-241b2c013167/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:56976 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:56914 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64170@0x68a0246a-SendThread(127.0.0.1:64170) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x635c2370-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:49140 [Waiting for operation #40] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:46024 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:57040 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_118379199_15 at /127.0.0.1:57062 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_816135261_15 at /127.0.0.1:56962 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_118379199_15 at /127.0.0.1:57034 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x635c2370-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64170@0x68a0246a-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) - Thread LEAK? -, OpenFileDescriptor=597 (was 590) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=245 (was 271), ProcessCount=168 (was 168), AvailableMemoryMB=3767 (was 4243) 2023-02-09 09:54:05,301 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=250, OpenFileDescriptor=597, MaxFileDescriptor=60000, SystemLoadAverage=245, ProcessCount=168, AvailableMemoryMB=3766 2023-02-09 09:54:05,302 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:05,302 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:05,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 09:54:05,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:05,309 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:54:05,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-02-09 09:54:05,309 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:05,310 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:05,311 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:54:05,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 09:54:05,316 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6 2023-02-09 09:54:05,317 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6 empty. 2023-02-09 09:54:05,318 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6 2023-02-09 09:54:05,318 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-02-09 09:54:05,330 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 09:54:05,332 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1f71bdb54b6da843ebd313ec10a31be6, NAME => 'testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:54:05,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:05,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 1f71bdb54b6da843ebd313ec10a31be6, disabling compactions & flushes 2023-02-09 09:54:05,346 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:05,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:05,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. after waiting 0 ms 2023-02-09 09:54:05,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:05,346 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:05,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 1f71bdb54b6da843ebd313ec10a31be6: Waiting for close lock at 1675936445346Disabling compacts and flushes for region at 1675936445346Disabling writes for close at 1675936445346Writing region close event to WAL at 1675936445346Closed at 1675936445346 2023-02-09 09:54:05,349 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:54:05,350 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675936445350"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936445350"}]},"ts":"1675936445350"} 2023-02-09 09:54:05,352 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 09:54:05,354 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:54:05,354 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936445354"}]},"ts":"1675936445354"} 2023-02-09 09:54:05,356 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-09 09:54:05,363 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1f71bdb54b6da843ebd313ec10a31be6, ASSIGN}] 2023-02-09 09:54:05,365 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1f71bdb54b6da843ebd313ec10a31be6, ASSIGN 2023-02-09 09:54:05,366 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1f71bdb54b6da843ebd313ec10a31be6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:54:05,411 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:05,411 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:05,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 09:54:05,518 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=1f71bdb54b6da843ebd313ec10a31be6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:05,518 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675936445518"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936445518"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936445518"}]},"ts":"1675936445518"} 2023-02-09 09:54:05,521 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 1f71bdb54b6da843ebd313ec10a31be6, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:05,613 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:05,613 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:05,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 09:54:05,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:05,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1f71bdb54b6da843ebd313ec10a31be6, NAME => 'testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:54:05,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 1f71bdb54b6da843ebd313ec10a31be6 2023-02-09 09:54:05,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:05,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1f71bdb54b6da843ebd313ec10a31be6 2023-02-09 09:54:05,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1f71bdb54b6da843ebd313ec10a31be6 2023-02-09 09:54:05,679 INFO [StoreOpener-1f71bdb54b6da843ebd313ec10a31be6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1f71bdb54b6da843ebd313ec10a31be6 2023-02-09 09:54:05,680 DEBUG [StoreOpener-1f71bdb54b6da843ebd313ec10a31be6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1 2023-02-09 09:54:05,681 DEBUG [StoreOpener-1f71bdb54b6da843ebd313ec10a31be6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1 2023-02-09 09:54:05,681 INFO [StoreOpener-1f71bdb54b6da843ebd313ec10a31be6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1f71bdb54b6da843ebd313ec10a31be6 columnFamilyName family1 2023-02-09 09:54:05,683 INFO [StoreOpener-1f71bdb54b6da843ebd313ec10a31be6-1] regionserver.HStore(310): Store=1f71bdb54b6da843ebd313ec10a31be6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:05,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6 2023-02-09 09:54:05,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6 2023-02-09 09:54:05,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1f71bdb54b6da843ebd313ec10a31be6 2023-02-09 09:54:05,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:54:05,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1f71bdb54b6da843ebd313ec10a31be6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74936711, jitterRate=0.1166440099477768}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:54:05,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1f71bdb54b6da843ebd313ec10a31be6: Running coprocessor pre-open hook at 1675936445678Writing region info on filesystem at 1675936445678Initializing all the Stores at 1675936445679 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675936445679Cleaning up temporary data from old regions at 1675936445685 (+6 ms)Cleaning up detritus from prior splits at 1675936445686 (+1 ms)Running coprocessor post-open hooks at 1675936445692 (+6 ms)Region opened successfully at 1675936445692 2023-02-09 09:54:05,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6., pid=25, masterSystemTime=1675936445673 2023-02-09 09:54:05,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:05,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:05,696 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=1f71bdb54b6da843ebd313ec10a31be6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:05,696 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675936445696"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936445696"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936445696"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936445696"}]},"ts":"1675936445696"} 2023-02-09 09:54:05,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-02-09 09:54:05,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 1f71bdb54b6da843ebd313ec10a31be6, server=jenkins-hbase4.apache.org,42719,1675936416061 in 177 msec 2023-02-09 09:54:05,702 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-02-09 09:54:05,703 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1f71bdb54b6da843ebd313ec10a31be6, ASSIGN in 337 msec 2023-02-09 09:54:05,703 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:54:05,703 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936445703"}]},"ts":"1675936445703"} 2023-02-09 09:54:05,705 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-09 09:54:05,708 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:54:05,709 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 403 msec 2023-02-09 09:54:05,915 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:05,915 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:05,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 09:54:05,916 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-02-09 09:54:05,937 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:05,937 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:05,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:05,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:05,940 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:05,940 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:05,940 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:05,940 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:05,940 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:05,940 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:05,941 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:05,941 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:05,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:05,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:05,942 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:05,942 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:05,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:05,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:05,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:05,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:05,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:05,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:05,943 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:05,943 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:05,943 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:05,943 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:05,943 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:05,943 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:05,943 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. started... 2023-02-09 09:54:05,944 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f71bdb54b6da843ebd313ec10a31be6 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 09:54:06,368 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130e5234d42bebd44da9be261a1fb665bbd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130e5234d42bebd44da9be261a1fb665bbd 2023-02-09 09:54:06,372 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/5f5660609f2c463dbb52e9feb6d55014 2023-02-09 09:54:06,378 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/5f5660609f2c463dbb52e9feb6d55014 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/5f5660609f2c463dbb52e9feb6d55014 2023-02-09 09:54:06,385 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/5f5660609f2c463dbb52e9feb6d55014, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-09 09:54:06,386 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 1f71bdb54b6da843ebd313ec10a31be6 in 442ms, sequenceid=5, compaction requested=false 2023-02-09 09:54:06,386 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:06,387 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:06,387 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:06,387 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:06,387 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:06,387 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,387 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:06,387 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:06,389 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,389 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:06,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:06,390 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,390 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:06,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:06,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:06,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:06,392 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:06,392 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f1e07a2[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:06,392 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:06,392 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,393 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,393 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:06,393 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:06,393 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:06,394 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,394 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:06,395 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:06,395 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,395 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:06,395 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:06,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:06,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:06,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:06,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:06,398 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:06,398 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:06,398 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:06,398 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,398 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:06,401 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,401 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:06,401 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:06,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:06,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:06,403 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:06,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,405 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:06,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,407 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:06,407 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:06,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:06,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:06,408 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,408 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,408 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,408 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,408 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:06,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:06,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:06,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:06,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:06,408 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:06,408 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,408 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:06,408 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:06,409 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,409 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,509 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:06,509 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:06,509 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:06,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:06,514 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:06,514 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:06,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:06,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:06,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,517 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:06,517 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:06,517 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:06,517 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:06,517 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,517 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,518 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:06,518 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:06,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:06,519 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:06,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,519 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:06,519 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:06,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,520 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,520 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:06,520 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:06,520 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:06,520 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:06,520 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:06,520 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:06,520 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:06,520 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:06,520 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. started... 2023-02-09 09:54:06,521 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f71bdb54b6da843ebd313ec10a31be6 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 09:54:06,936 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128227ec5be46034a0f86d88160bfd8c572 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128227ec5be46034a0f86d88160bfd8c572 2023-02-09 09:54:07,343 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/bde60f31efe44b36b68e1da0f97ec3f9 2023-02-09 09:54:07,350 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/bde60f31efe44b36b68e1da0f97ec3f9 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/bde60f31efe44b36b68e1da0f97ec3f9 2023-02-09 09:54:07,356 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/bde60f31efe44b36b68e1da0f97ec3f9, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 09:54:07,357 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 1f71bdb54b6da843ebd313ec10a31be6 in 836ms, sequenceid=9, compaction requested=false 2023-02-09 09:54:07,358 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:07,358 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:07,358 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:07,358 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:07,358 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,358 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:07,358 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:07,368 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,368 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:07,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:07,368 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:07,369 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:07,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:07,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:07,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:07,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32c7faea[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:07,370 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:07,370 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,372 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,372 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:07,372 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:07,372 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:07,372 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,372 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:07,373 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,373 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:07,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:07,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:07,373 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:07,374 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:07,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:07,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:07,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:07,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:07,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:07,376 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:07,376 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:07,376 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:07,376 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,376 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:07,379 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,379 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,379 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:07,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:07,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:07,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:07,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:07,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:07,381 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:07,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,382 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:07,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,385 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,385 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:07,385 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:07,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:07,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:07,385 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:07,385 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,385 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:07,386 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:07,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:07,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:07,385 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,386 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:07,386 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,386 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:07,386 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:07,386 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,486 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:07,486 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:07,486 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:07,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:07,491 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:07,491 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:07,492 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:07,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:07,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,493 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:07,493 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:07,493 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:07,494 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:07,494 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,494 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,495 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,495 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:07,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:07,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:07,495 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:07,495 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:07,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,496 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:07,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,496 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,496 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:07,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:07,497 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:07,497 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:07,497 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:07,497 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:07,497 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:07,497 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:07,497 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. started... 2023-02-09 09:54:07,497 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f71bdb54b6da843ebd313ec10a31be6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:07,914 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120105df6cf3a79142d6bb74f47e80c98ef0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120105df6cf3a79142d6bb74f47e80c98ef0 2023-02-09 09:54:08,320 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/4f79e798e724468fa88db5446369a63c 2023-02-09 09:54:08,328 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/4f79e798e724468fa88db5446369a63c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/4f79e798e724468fa88db5446369a63c 2023-02-09 09:54:08,334 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/4f79e798e724468fa88db5446369a63c, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 09:54:08,338 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f71bdb54b6da843ebd313ec10a31be6 in 841ms, sequenceid=13, compaction requested=true 2023-02-09 09:54:08,338 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:08,338 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:08,338 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:08,339 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:08,339 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,339 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:08,339 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:08,340 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,340 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:08,341 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,341 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:08,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:08,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b06d3b0[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:08,343 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:08,343 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,346 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,346 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:08,346 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:08,346 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:08,346 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,346 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:08,348 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,349 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:08,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:08,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:08,349 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:08,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:08,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:08,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:08,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:08,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:08,352 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:08,352 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:08,352 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:08,353 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,353 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:08,354 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,354 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:08,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:08,354 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:08,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:08,355 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:08,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:08,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:08,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,357 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:08,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,368 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,368 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:08,368 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:08,368 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:08,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:08,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:08,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:08,368 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:08,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:08,369 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,369 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,369 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,369 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,369 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,369 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:08,369 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:08,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:08,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:08,469 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:08,470 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:08,470 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:08,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:08,474 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:08,474 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:08,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:08,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:08,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,477 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:08,477 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:08,477 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:08,477 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:08,477 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,477 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,478 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:08,478 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:08,479 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,479 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,480 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,480 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:08,480 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:08,480 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. started... 2023-02-09 09:54:08,480 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f71bdb54b6da843ebd313ec10a31be6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:08,896 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205a8282d67eca740c0a00b6aec476b52dc to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205a8282d67eca740c0a00b6aec476b52dc 2023-02-09 09:54:08,905 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/160f04339eed42649e5e370722d3d62b 2023-02-09 09:54:08,923 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/160f04339eed42649e5e370722d3d62b as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/160f04339eed42649e5e370722d3d62b 2023-02-09 09:54:08,934 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/160f04339eed42649e5e370722d3d62b, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 09:54:08,936 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f71bdb54b6da843ebd313ec10a31be6 in 456ms, sequenceid=17, compaction requested=true 2023-02-09 09:54:08,936 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:08,936 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:08,938 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,938 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:08,939 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,939 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:08,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:08,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:08,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:08,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28930273[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:08,943 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:08,943 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,944 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,945 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:08,945 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:08,945 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:08,946 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,946 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:08,948 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:08,948 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,948 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:08,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:08,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:08,949 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:08,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:08,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:08,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:08,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:08,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:08,952 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:08,952 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:08,952 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:08,952 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,952 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:08,954 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,954 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:08,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,954 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:08,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:08,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:08,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:08,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:08,955 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:08,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,957 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:08,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:08,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:08,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:08,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:08,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:08,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:08,961 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:08,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:08,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:08,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:08,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:08,962 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:08,962 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:09,062 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:09,063 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:09,063 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:09,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:09,068 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:09,068 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:09,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:09,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:09,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,071 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:09,072 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:09,072 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:09,072 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:09,072 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,072 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,073 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:09,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,074 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,075 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,075 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:09,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:09,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,076 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:09,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,076 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:09,076 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:09,076 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:09,076 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:09,076 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:09,077 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:09,077 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. started... 2023-02-09 09:54:09,077 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f71bdb54b6da843ebd313ec10a31be6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:09,509 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512284263e34c46c345708ce6ccb395c3cf84 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512284263e34c46c345708ce6ccb395c3cf84 2023-02-09 09:54:09,518 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/8694769752a74d14a4b43ab24b9f0d7a 2023-02-09 09:54:09,524 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/8694769752a74d14a4b43ab24b9f0d7a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/8694769752a74d14a4b43ab24b9f0d7a 2023-02-09 09:54:09,530 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/8694769752a74d14a4b43ab24b9f0d7a, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 09:54:09,531 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f71bdb54b6da843ebd313ec10a31be6 in 454ms, sequenceid=21, compaction requested=true 2023-02-09 09:54:09,531 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:09,531 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:09,531 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:09,531 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:09,531 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,531 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:09,531 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:09,533 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,533 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:09,534 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:09,534 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79f23d4[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:09,535 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:09,536 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,537 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,537 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:09,537 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:09,537 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:09,537 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,537 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:09,539 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:09,539 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,539 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:09,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:09,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:09,539 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:09,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:09,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:09,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:09,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:09,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:09,543 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:09,543 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:09,543 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:09,543 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,543 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:09,545 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,545 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,545 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:09,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:09,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:09,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:09,546 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:09,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:09,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:09,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,547 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:09,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,555 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,555 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,556 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:09,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:09,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:09,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:09,556 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:09,556 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:09,556 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:09,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:09,556 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,556 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,556 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:09,556 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:09,556 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,557 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:09,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:09,657 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:09,657 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:09,657 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:09,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:09,662 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:09,662 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:09,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:09,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:09,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,666 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:09,666 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:09,666 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:09,666 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:09,667 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,667 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,668 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:09,668 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:09,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:09,668 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:09,668 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:09,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,669 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,669 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:09,669 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:09,670 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:09,670 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:09,670 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:09,670 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:09,670 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:09,670 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:09,670 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. started... 2023-02-09 09:54:09,670 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f71bdb54b6da843ebd313ec10a31be6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:10,084 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a8cca045eea146178bb8247b9404dbe6 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231a8cca045eea146178bb8247b9404dbe6 2023-02-09 09:54:10,490 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/480ca5446d6841d2b909ae351fea5a85 2023-02-09 09:54:10,496 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/480ca5446d6841d2b909ae351fea5a85 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/480ca5446d6841d2b909ae351fea5a85 2023-02-09 09:54:10,500 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/480ca5446d6841d2b909ae351fea5a85, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 09:54:10,501 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f71bdb54b6da843ebd313ec10a31be6 in 831ms, sequenceid=25, compaction requested=true 2023-02-09 09:54:10,501 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:10,501 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:10,501 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:10,501 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:10,502 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,502 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:10,502 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:10,503 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,503 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:10,503 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:10,504 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:10,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:10,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:10,505 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:10,505 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a028967[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:10,505 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:10,505 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,506 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:10,506 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,506 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:10,511 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,511 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:10,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:10,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:10,511 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:10,511 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:10,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:10,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:10,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:10,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:10,513 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:10,513 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:10,513 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:10,514 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,514 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:10,515 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,515 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:10,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,515 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:10,515 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:10,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:10,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:10,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,517 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:10,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,519 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,519 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,519 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,519 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,519 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,519 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:10,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:10,519 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:10,519 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:10,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:10,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:10,519 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,519 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:10,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:10,519 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:10,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:10,520 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:10,520 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:10,620 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:10,620 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:10,620 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:10,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:10,625 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:10,625 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:10,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:10,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:10,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,627 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:10,627 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:10,627 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:10,627 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:10,627 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,628 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,629 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:10,629 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:10,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:10,629 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:10,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,629 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:10,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:10,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:10,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:10,630 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:10,630 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:10,630 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:10,631 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:10,631 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:10,631 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:10,631 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. started... 2023-02-09 09:54:10,631 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f71bdb54b6da843ebd313ec10a31be6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:11,045 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101fbdd852208774ed39067795a68a1093c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101fbdd852208774ed39067795a68a1093c 2023-02-09 09:54:11,206 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:54:11,451 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/d7af44dd747f49eca21d47ed95d935d4 2023-02-09 09:54:11,457 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/d7af44dd747f49eca21d47ed95d935d4 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/d7af44dd747f49eca21d47ed95d935d4 2023-02-09 09:54:11,461 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/d7af44dd747f49eca21d47ed95d935d4, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 09:54:11,462 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f71bdb54b6da843ebd313ec10a31be6 in 831ms, sequenceid=29, compaction requested=true 2023-02-09 09:54:11,463 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:11,463 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:11,463 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:11,463 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:11,463 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,463 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:11,463 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:11,464 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,464 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:11,465 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,465 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:11,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:11,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:11,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79440c4[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:11,466 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:11,467 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,468 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,468 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:11,468 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:11,468 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:11,468 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,468 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:11,469 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:11,469 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:11,469 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:11,469 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:11,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:11,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:11,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:11,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:11,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:11,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:11,473 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:11,473 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:11,473 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:11,473 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,473 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:11,475 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,475 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:11,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,475 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:11,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:11,476 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:11,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,477 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:11,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,479 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,479 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:11,479 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:11,480 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:11,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:11,480 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:11,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:11,479 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:11,479 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:11,480 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,480 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:11,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:11,480 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:11,480 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,480 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:11,480 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,581 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:11,581 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:11,581 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:11,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:11,586 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:11,586 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:11,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:11,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:11,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,589 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:11,589 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:11,589 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:11,589 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:11,589 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,589 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,591 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:11,591 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:11,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:11,591 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:11,591 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:11,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,592 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:11,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,592 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:11,592 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:11,592 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:11,592 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:11,592 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:11,593 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:11,593 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:11,593 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:11,593 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. started... 2023-02-09 09:54:11,593 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f71bdb54b6da843ebd313ec10a31be6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:11,607 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010307ab6d7a4deb4c5aabd4dda9f0c18c60 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010307ab6d7a4deb4c5aabd4dda9f0c18c60 2023-02-09 09:54:12,014 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/50f38bfcc7d841bbbdaa4f33b57884b4 2023-02-09 09:54:12,019 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/50f38bfcc7d841bbbdaa4f33b57884b4 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/50f38bfcc7d841bbbdaa4f33b57884b4 2023-02-09 09:54:12,024 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/50f38bfcc7d841bbbdaa4f33b57884b4, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 09:54:12,025 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f71bdb54b6da843ebd313ec10a31be6 in 432ms, sequenceid=33, compaction requested=true 2023-02-09 09:54:12,025 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:12,025 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:12,025 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:12,025 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:12,025 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,025 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:12,025 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:12,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,027 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:12,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:12,027 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,027 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:12,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:12,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:12,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:12,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:12,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37245adf[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:12,029 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:12,029 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,031 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,031 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:12,031 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:12,031 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,031 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:12,031 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:12,033 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:12,033 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:12,033 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,033 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:12,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:12,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:12,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:12,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:12,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:12,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:12,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:12,035 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:12,035 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:12,035 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:12,035 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,035 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:12,037 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,037 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:12,037 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:12,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:12,037 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:12,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,038 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:12,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,041 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,041 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,041 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:12,041 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:12,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:12,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:12,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:12,041 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:12,041 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:12,041 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,041 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:12,041 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:12,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:12,041 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:12,042 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,042 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,141 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:12,142 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:12,142 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:12,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:12,146 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:12,146 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:12,147 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:12,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:12,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,149 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:12,149 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:12,149 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:12,149 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:12,149 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,149 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,151 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,151 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:12,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:12,152 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,152 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,152 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:12,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,153 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,153 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:12,153 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,153 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:12,153 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:12,153 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:12,153 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:12,153 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:12,153 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:12,153 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. started... 2023-02-09 09:54:12,153 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f71bdb54b6da843ebd313ec10a31be6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:12,567 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907fc3d3115ab124effa17f9e812d17872d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907fc3d3115ab124effa17f9e812d17872d 2023-02-09 09:54:12,974 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/7c01a8af6e1f4f1090bbc50a284727f8 2023-02-09 09:54:12,979 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/7c01a8af6e1f4f1090bbc50a284727f8 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/7c01a8af6e1f4f1090bbc50a284727f8 2023-02-09 09:54:12,984 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/7c01a8af6e1f4f1090bbc50a284727f8, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 09:54:12,985 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f71bdb54b6da843ebd313ec10a31be6 in 832ms, sequenceid=37, compaction requested=true 2023-02-09 09:54:12,985 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:12,985 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:12,985 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:12,985 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:12,985 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,985 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:12,985 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:12,987 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,987 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:12,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:12,987 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,987 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:12,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:12,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:12,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:12,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:12,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49db5e6c[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:12,989 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:12,989 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,992 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,992 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:12,992 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:12,992 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,992 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:12,992 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:12,995 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:12,995 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:12,995 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,995 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:12,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:12,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:12,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:12,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:12,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:12,997 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:12,997 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:12,997 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:12,997 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,997 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:12,999 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,999 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,999 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:12,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:12,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:12,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:12,999 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:12,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:12,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:12,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:13,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:13,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,000 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,003 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,003 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:13,003 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:13,003 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:13,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:13,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:13,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:13,003 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:13,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:13,003 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,004 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,004 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:13,004 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,004 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:13,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:13,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:13,004 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,004 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,104 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:13,104 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:13,104 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:13,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:13,109 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:13,109 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:13,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:13,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:13,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,111 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:13,112 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:13,112 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:13,112 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:13,112 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,112 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,113 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:13,113 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:13,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:13,114 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,114 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:13,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,115 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:13,115 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:13,115 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:13,115 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:13,115 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:13,115 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:13,115 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. started... 2023-02-09 09:54:13,115 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f71bdb54b6da843ebd313ec10a31be6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:13,529 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e441289220424f36b8c40d8d6f901612 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120e441289220424f36b8c40d8d6f901612 2023-02-09 09:54:13,935 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/0114d00096a84b0f907c6dd9a20d1aca 2023-02-09 09:54:13,940 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/0114d00096a84b0f907c6dd9a20d1aca as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/0114d00096a84b0f907c6dd9a20d1aca 2023-02-09 09:54:13,945 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/0114d00096a84b0f907c6dd9a20d1aca, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 09:54:13,945 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f71bdb54b6da843ebd313ec10a31be6 in 830ms, sequenceid=41, compaction requested=true 2023-02-09 09:54:13,946 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:13,946 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:13,946 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:13,946 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:13,946 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,946 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:13,946 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:13,948 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,948 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:13,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:13,948 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,948 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:13,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:13,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:13,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:13,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:13,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@578caf81[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:13,950 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:13,950 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,951 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,951 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:13,951 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:13,951 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:13,951 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,951 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:13,952 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:13,952 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,952 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:13,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:13,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:13,952 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:13,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:13,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:13,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:13,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:13,954 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:13,954 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:13,954 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:13,954 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:13,955 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,955 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:13,956 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,956 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:13,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:13,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,956 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:13,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:13,956 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:13,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:13,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:13,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,957 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:13,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,960 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,960 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:13,960 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:13,960 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 09:54:13,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:13,960 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:13,960 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:13,960 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:13,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:13,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,961 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:13,961 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:13,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:13,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:13,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:13,961 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:14,061 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:14,061 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:14,061 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:14,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:14,063 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-02-09 09:54:14,065 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:54:14,066 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:54:14,067 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:14,067 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937054067 2023-02-09 09:54:14,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 09:54:14,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-09 09:54:14,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:14,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x757dee88 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:54:14,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@75511c20, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:54:14,080 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:14,156 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ad472802b1604b98b23dd90714d36888 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ad472802b1604b98b23dd90714d36888 2023-02-09 09:54:14,157 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:14,173 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:54:14,174 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58898, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:54:14,183 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:14,183 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:14,184 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:54:14,186 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60798, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:54:14,210 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/845cb151ec0248909c917d646af68a21 first=Optional[r3] last=Optional[r6] 2023-02-09 09:54:14,211 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/845cb151ec0248909c917d646af68a21}] 2023-02-09 09:54:14,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/845cb151ec0248909c917d646af68a21 for inclusion in 1f71bdb54b6da843ebd313ec10a31be6/family1 2023-02-09 09:54:14,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 09:54:14,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:54:14,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:14,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/845cb151ec0248909c917d646af68a21 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q0qkaei5il2v6lc84pdnmcnr6j3lm7urvbbkbrd6fa9q7smjleckb66qsb6dcj8t/family1/845cb151ec0248909c917d646af68a21 2023-02-09 09:54:14,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q0qkaei5il2v6lc84pdnmcnr6j3lm7urvbbkbrd6fa9q7smjleckb66qsb6dcj8t/family1/845cb151ec0248909c917d646af68a21 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/ea9e60df9a404ce78b09ff7cc19fc1f0_SeqId_44_ 2023-02-09 09:54:14,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q0qkaei5il2v6lc84pdnmcnr6j3lm7urvbbkbrd6fa9q7smjleckb66qsb6dcj8t/family1/845cb151ec0248909c917d646af68a21 into 1f71bdb54b6da843ebd313ec10a31be6/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/ea9e60df9a404ce78b09ff7cc19fc1f0_SeqId_44_ - updating store file list. 2023-02-09 09:54:14,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/ea9e60df9a404ce78b09ff7cc19fc1f0_SeqId_44_ into 1f71bdb54b6da843ebd313ec10a31be6/family1 2023-02-09 09:54:14,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q0qkaei5il2v6lc84pdnmcnr6j3lm7urvbbkbrd6fa9q7smjleckb66qsb6dcj8t/family1/845cb151ec0248909c917d646af68a21 into 1f71bdb54b6da843ebd313ec10a31be6/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/ea9e60df9a404ce78b09ff7cc19fc1f0_SeqId_44_) 2023-02-09 09:54:14,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q0qkaei5il2v6lc84pdnmcnr6j3lm7urvbbkbrd6fa9q7smjleckb66qsb6dcj8t/family1/845cb151ec0248909c917d646af68a21 2023-02-09 09:54:14,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 09:54:14,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 09:54:14,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): 1f71bdb54b6da843ebd313ec10a31be6/family1 is initiating minor compaction 2023-02-09 09:54:14,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6087a59c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 09:54:14,236 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1f71bdb54b6da843ebd313ec10a31be6/family1 in testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:54:14,236 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/bde60f31efe44b36b68e1da0f97ec3f9, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/4f79e798e724468fa88db5446369a63c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/160f04339eed42649e5e370722d3d62b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/8694769752a74d14a4b43ab24b9f0d7a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/480ca5446d6841d2b909ae351fea5a85, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/d7af44dd747f49eca21d47ed95d935d4, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/50f38bfcc7d841bbbdaa4f33b57884b4, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/7c01a8af6e1f4f1090bbc50a284727f8, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/0114d00096a84b0f907c6dd9a20d1aca, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/ea9e60df9a404ce78b09ff7cc19fc1f0_SeqId_44_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp, totalSize=50.0 K 2023-02-09 09:54:14,236 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting bde60f31efe44b36b68e1da0f97ec3f9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 09:54:14,237 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting 4f79e798e724468fa88db5446369a63c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 09:54:14,237 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting 160f04339eed42649e5e370722d3d62b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 09:54:14,237 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting 8694769752a74d14a4b43ab24b9f0d7a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 09:54:14,237 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting 480ca5446d6841d2b909ae351fea5a85, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 09:54:14,237 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting d7af44dd747f49eca21d47ed95d935d4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 09:54:14,237 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting 50f38bfcc7d841bbbdaa4f33b57884b4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 09:54:14,237 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting 7c01a8af6e1f4f1090bbc50a284727f8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 09:54:14,237 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting 0114d00096a84b0f907c6dd9a20d1aca, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 09:54:14,237 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting ea9e60df9a404ce78b09ff7cc19fc1f0_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 09:54:14,239 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:14,240 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:14,242 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512284263e34c46c345708ce6ccb395c3cf84 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512284263e34c46c345708ce6ccb395c3cf84 2023-02-09 09:54:14,243 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231a8cca045eea146178bb8247b9404dbe6 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231a8cca045eea146178bb8247b9404dbe6 2023-02-09 09:54:14,245 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101fbdd852208774ed39067795a68a1093c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101fbdd852208774ed39067795a68a1093c 2023-02-09 09:54:14,246 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010307ab6d7a4deb4c5aabd4dda9f0c18c60 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010307ab6d7a4deb4c5aabd4dda9f0c18c60 2023-02-09 09:54:14,246 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-09 09:54:14,246 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 09:54:14,247 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:14,247 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:14,247 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:14,247 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:14,247 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:14,247 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:14,247 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:14,247 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:14,288 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] throttle.PressureAwareThroughputController(145): 1f71bdb54b6da843ebd313ec10a31be6#family1#compaction#34 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:54:14,315 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512052164f917afce451c8e4f621e794f4a5b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512052164f917afce451c8e4f621e794f4a5b 2023-02-09 09:54:14,317 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:14,342 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:14,342 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:14,353 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b7a69b108da1407aa6c3df04d5f906ee first=Optional[r1] last=Optional[r2] 2023-02-09 09:54:14,354 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b7a69b108da1407aa6c3df04d5f906ee}] 2023-02-09 09:54:14,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b7a69b108da1407aa6c3df04d5f906ee for inclusion in 1f71bdb54b6da843ebd313ec10a31be6/family1 2023-02-09 09:54:14,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-09 09:54:14,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:54:14,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:14,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b7a69b108da1407aa6c3df04d5f906ee to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q59p40e2hv25q3ud7n7n9mrtngnl5m398kb2p0enfg6jd5jdhlnbpqcn55p7g59e/family1/b7a69b108da1407aa6c3df04d5f906ee 2023-02-09 09:54:14,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q59p40e2hv25q3ud7n7n9mrtngnl5m398kb2p0enfg6jd5jdhlnbpqcn55p7g59e/family1/b7a69b108da1407aa6c3df04d5f906ee as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/0b9248c5f64e4d208768faef564a2fae_SeqId_46_ 2023-02-09 09:54:14,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q59p40e2hv25q3ud7n7n9mrtngnl5m398kb2p0enfg6jd5jdhlnbpqcn55p7g59e/family1/b7a69b108da1407aa6c3df04d5f906ee into 1f71bdb54b6da843ebd313ec10a31be6/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/0b9248c5f64e4d208768faef564a2fae_SeqId_46_ - updating store file list. 2023-02-09 09:54:14,375 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/0b9248c5f64e4d208768faef564a2fae_SeqId_46_ into 1f71bdb54b6da843ebd313ec10a31be6/family1 2023-02-09 09:54:14,375 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q59p40e2hv25q3ud7n7n9mrtngnl5m398kb2p0enfg6jd5jdhlnbpqcn55p7g59e/family1/b7a69b108da1407aa6c3df04d5f906ee into 1f71bdb54b6da843ebd313ec10a31be6/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/0b9248c5f64e4d208768faef564a2fae_SeqId_46_) 2023-02-09 09:54:14,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q59p40e2hv25q3ud7n7n9mrtngnl5m398kb2p0enfg6jd5jdhlnbpqcn55p7g59e/family1/b7a69b108da1407aa6c3df04d5f906ee 2023-02-09 09:54:14,385 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:14,387 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:14,389 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120105df6cf3a79142d6bb74f47e80c98ef0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120105df6cf3a79142d6bb74f47e80c98ef0 2023-02-09 09:54:14,391 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205a8282d67eca740c0a00b6aec476b52dc to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205a8282d67eca740c0a00b6aec476b52dc 2023-02-09 09:54:14,391 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-02-09 09:54:14,391 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 09:54:14,391 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:14,392 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:14,392 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:14,393 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:54:14,396 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 329 msec 2023-02-09 09:54:14,525 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=256 (was 250) - Thread LEAK? -, OpenFileDescriptor=619 (was 597) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=303 (was 245) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3626 (was 3766) 2023-02-09 09:54:14,539 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=256, OpenFileDescriptor=619, MaxFileDescriptor=60000, SystemLoadAverage=303, ProcessCount=168, AvailableMemoryMB=3626 2023-02-09 09:54:14,540 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:14,540 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:14,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 09:54:14,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:14,546 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:54:14,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-02-09 09:54:14,546 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:14,546 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:14,547 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:54:14,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 09:54:14,549 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:54:14,549 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:54:14,549 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846 2023-02-09 09:54:14,550 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef empty. 2023-02-09 09:54:14,550 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d empty. 2023-02-09 09:54:14,550 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846 empty. 2023-02-09 09:54:14,550 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:54:14,550 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:54:14,550 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846 2023-02-09 09:54:14,550 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-02-09 09:54:14,647 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:14,648 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:14,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 09:54:14,737 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/.tmp/family1/8e9dfb5839234d0590cb282cc705f3e4 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/8e9dfb5839234d0590cb282cc705f3e4 2023-02-09 09:54:14,750 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 1f71bdb54b6da843ebd313ec10a31be6/family1 of 1f71bdb54b6da843ebd313ec10a31be6 into 8e9dfb5839234d0590cb282cc705f3e4(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:54:14,751 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1f71bdb54b6da843ebd313ec10a31be6: 2023-02-09 09:54:14,751 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1675936454236; duration=0sec 2023-02-09 09:54:14,751 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:14,849 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:14,850 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:14,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/bde60f31efe44b36b68e1da0f97ec3f9, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/4f79e798e724468fa88db5446369a63c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/160f04339eed42649e5e370722d3d62b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/8694769752a74d14a4b43ab24b9f0d7a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/480ca5446d6841d2b909ae351fea5a85, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/d7af44dd747f49eca21d47ed95d935d4, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/50f38bfcc7d841bbbdaa4f33b57884b4, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/7c01a8af6e1f4f1090bbc50a284727f8, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/0114d00096a84b0f907c6dd9a20d1aca, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/ea9e60df9a404ce78b09ff7cc19fc1f0_SeqId_44_] to archive 2023-02-09 09:54:14,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 09:54:14,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:14,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/bde60f31efe44b36b68e1da0f97ec3f9 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/bde60f31efe44b36b68e1da0f97ec3f9 2023-02-09 09:54:14,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/4f79e798e724468fa88db5446369a63c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/4f79e798e724468fa88db5446369a63c 2023-02-09 09:54:14,857 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/160f04339eed42649e5e370722d3d62b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/160f04339eed42649e5e370722d3d62b 2023-02-09 09:54:14,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/8694769752a74d14a4b43ab24b9f0d7a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/8694769752a74d14a4b43ab24b9f0d7a 2023-02-09 09:54:14,860 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/480ca5446d6841d2b909ae351fea5a85 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/480ca5446d6841d2b909ae351fea5a85 2023-02-09 09:54:14,863 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/d7af44dd747f49eca21d47ed95d935d4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/d7af44dd747f49eca21d47ed95d935d4 2023-02-09 09:54:14,864 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/50f38bfcc7d841bbbdaa4f33b57884b4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/50f38bfcc7d841bbbdaa4f33b57884b4 2023-02-09 09:54:14,866 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/7c01a8af6e1f4f1090bbc50a284727f8 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/7c01a8af6e1f4f1090bbc50a284727f8 2023-02-09 09:54:14,867 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/0114d00096a84b0f907c6dd9a20d1aca to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/0114d00096a84b0f907c6dd9a20d1aca 2023-02-09 09:54:14,868 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/ea9e60df9a404ce78b09ff7cc19fc1f0_SeqId_44_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/family1/ea9e60df9a404ce78b09ff7cc19fc1f0_SeqId_44_ 2023-02-09 09:54:14,959 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-02-09 09:54:14,961 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7671): creating {ENCODED => 9e302a2c16508a1b912e987cb0942846, NAME => 'testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:54:14,961 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7671): creating {ENCODED => 1b21e89c0550bd11bb849ead0d0c4c3d, NAME => 'testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:54:14,961 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7671): creating {ENCODED => 086d69d7e75a47257d3c163dd2bc28ef, NAME => 'testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:54:14,970 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:14,970 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1603): Closing 1b21e89c0550bd11bb849ead0d0c4c3d, disabling compactions & flushes 2023-02-09 09:54:14,970 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:14,970 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:14,970 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. after waiting 0 ms 2023-02-09 09:54:14,970 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:14,970 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:14,970 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1557): Region close journal for 1b21e89c0550bd11bb849ead0d0c4c3d: Waiting for close lock at 1675936454970Disabling compacts and flushes for region at 1675936454970Disabling writes for close at 1675936454970Writing region close event to WAL at 1675936454970Closed at 1675936454970 2023-02-09 09:54:15,151 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:15,152 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:15,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 09:54:15,370 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:15,370 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1603): Closing 086d69d7e75a47257d3c163dd2bc28ef, disabling compactions & flushes 2023-02-09 09:54:15,370 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:15,370 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:15,370 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. after waiting 0 ms 2023-02-09 09:54:15,370 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:15,370 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:15,370 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1557): Region close journal for 086d69d7e75a47257d3c163dd2bc28ef: Waiting for close lock at 1675936455370Disabling compacts and flushes for region at 1675936455370Disabling writes for close at 1675936455370Writing region close event to WAL at 1675936455370Closed at 1675936455370 2023-02-09 09:54:15,370 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:15,370 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1603): Closing 9e302a2c16508a1b912e987cb0942846, disabling compactions & flushes 2023-02-09 09:54:15,371 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:15,371 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:15,371 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. after waiting 0 ms 2023-02-09 09:54:15,371 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:15,371 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:15,371 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1557): Region close journal for 9e302a2c16508a1b912e987cb0942846: Waiting for close lock at 1675936455370Disabling compacts and flushes for region at 1675936455370Disabling writes for close at 1675936455371 (+1 ms)Writing region close event to WAL at 1675936455371Closed at 1675936455371 2023-02-09 09:54:15,374 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:54:15,375 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675936455374"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936455374"}]},"ts":"1675936455374"} 2023-02-09 09:54:15,375 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675936455374"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936455374"}]},"ts":"1675936455374"} 2023-02-09 09:54:15,375 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675936455374"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936455374"}]},"ts":"1675936455374"} 2023-02-09 09:54:15,378 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 09:54:15,379 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:54:15,379 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936455379"}]},"ts":"1675936455379"} 2023-02-09 09:54:15,381 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-02-09 09:54:15,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=086d69d7e75a47257d3c163dd2bc28ef, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1b21e89c0550bd11bb849ead0d0c4c3d, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=9e302a2c16508a1b912e987cb0942846, ASSIGN}] 2023-02-09 09:54:15,388 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=9e302a2c16508a1b912e987cb0942846, ASSIGN 2023-02-09 09:54:15,389 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=086d69d7e75a47257d3c163dd2bc28ef, ASSIGN 2023-02-09 09:54:15,389 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1b21e89c0550bd11bb849ead0d0c4c3d, ASSIGN 2023-02-09 09:54:15,391 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=9e302a2c16508a1b912e987cb0942846, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:54:15,391 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=086d69d7e75a47257d3c163dd2bc28ef, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:54:15,391 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1b21e89c0550bd11bb849ead0d0c4c3d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:54:15,543 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=9e302a2c16508a1b912e987cb0942846, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:15,543 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=1b21e89c0550bd11bb849ead0d0c4c3d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:15,543 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=086d69d7e75a47257d3c163dd2bc28ef, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:15,543 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675936455543"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936455543"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936455543"}]},"ts":"1675936455543"} 2023-02-09 09:54:15,543 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675936455543"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936455543"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936455543"}]},"ts":"1675936455543"} 2023-02-09 09:54:15,543 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675936455543"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936455543"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936455543"}]},"ts":"1675936455543"} 2023-02-09 09:54:15,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=30, state=RUNNABLE; OpenRegionProcedure 9e302a2c16508a1b912e987cb0942846, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:15,546 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=29, state=RUNNABLE; OpenRegionProcedure 1b21e89c0550bd11bb849ead0d0c4c3d, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:15,547 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=28, state=RUNNABLE; OpenRegionProcedure 086d69d7e75a47257d3c163dd2bc28ef, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:15,653 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:15,653 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:15,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 09:54:15,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:15,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9e302a2c16508a1b912e987cb0942846, NAME => 'testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 09:54:15,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 9e302a2c16508a1b912e987cb0942846 2023-02-09 09:54:15,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:15,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9e302a2c16508a1b912e987cb0942846 2023-02-09 09:54:15,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9e302a2c16508a1b912e987cb0942846 2023-02-09 09:54:15,705 INFO [StoreOpener-9e302a2c16508a1b912e987cb0942846-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9e302a2c16508a1b912e987cb0942846 2023-02-09 09:54:15,706 DEBUG [StoreOpener-9e302a2c16508a1b912e987cb0942846-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1 2023-02-09 09:54:15,707 DEBUG [StoreOpener-9e302a2c16508a1b912e987cb0942846-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1 2023-02-09 09:54:15,707 INFO [StoreOpener-9e302a2c16508a1b912e987cb0942846-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9e302a2c16508a1b912e987cb0942846 columnFamilyName family1 2023-02-09 09:54:15,709 INFO [StoreOpener-9e302a2c16508a1b912e987cb0942846-1] regionserver.HStore(310): Store=9e302a2c16508a1b912e987cb0942846/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:15,709 INFO [StoreOpener-9e302a2c16508a1b912e987cb0942846-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9e302a2c16508a1b912e987cb0942846 2023-02-09 09:54:15,711 DEBUG [StoreOpener-9e302a2c16508a1b912e987cb0942846-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family2 2023-02-09 09:54:15,711 DEBUG [StoreOpener-9e302a2c16508a1b912e987cb0942846-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family2 2023-02-09 09:54:15,711 INFO [StoreOpener-9e302a2c16508a1b912e987cb0942846-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9e302a2c16508a1b912e987cb0942846 columnFamilyName family2 2023-02-09 09:54:15,713 INFO [StoreOpener-9e302a2c16508a1b912e987cb0942846-1] regionserver.HStore(310): Store=9e302a2c16508a1b912e987cb0942846/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:15,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846 2023-02-09 09:54:15,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846 2023-02-09 09:54:15,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:54:15,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9e302a2c16508a1b912e987cb0942846 2023-02-09 09:54:15,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:54:15,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9e302a2c16508a1b912e987cb0942846; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60810504, jitterRate=-0.09385287761688232}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:54:15,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9e302a2c16508a1b912e987cb0942846: Running coprocessor pre-open hook at 1675936455704Writing region info on filesystem at 1675936455704Initializing all the Stores at 1675936455705 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936455705Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936455705Cleaning up temporary data from old regions at 1675936455715 (+10 ms)Cleaning up detritus from prior splits at 1675936455715Running coprocessor post-open hooks at 1675936455721 (+6 ms)Region opened successfully at 1675936455721 2023-02-09 09:54:15,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846., pid=31, masterSystemTime=1675936455699 2023-02-09 09:54:15,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:15,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:15,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:15,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1b21e89c0550bd11bb849ead0d0c4c3d, NAME => 'testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 09:54:15,725 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=9e302a2c16508a1b912e987cb0942846, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:15,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:54:15,725 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675936455725"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936455725"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936455725"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936455725"}]},"ts":"1675936455725"} 2023-02-09 09:54:15,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:15,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:54:15,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:54:15,727 INFO [StoreOpener-1b21e89c0550bd11bb849ead0d0c4c3d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:54:15,728 DEBUG [StoreOpener-1b21e89c0550bd11bb849ead0d0c4c3d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d/family1 2023-02-09 09:54:15,729 DEBUG [StoreOpener-1b21e89c0550bd11bb849ead0d0c4c3d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d/family1 2023-02-09 09:54:15,729 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=30 2023-02-09 09:54:15,729 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=30, state=SUCCESS; OpenRegionProcedure 9e302a2c16508a1b912e987cb0942846, server=jenkins-hbase4.apache.org,42719,1675936416061 in 181 msec 2023-02-09 09:54:15,729 INFO [StoreOpener-1b21e89c0550bd11bb849ead0d0c4c3d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b21e89c0550bd11bb849ead0d0c4c3d columnFamilyName family1 2023-02-09 09:54:15,731 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=9e302a2c16508a1b912e987cb0942846, ASSIGN in 345 msec 2023-02-09 09:54:15,731 INFO [StoreOpener-1b21e89c0550bd11bb849ead0d0c4c3d-1] regionserver.HStore(310): Store=1b21e89c0550bd11bb849ead0d0c4c3d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:15,732 INFO [StoreOpener-1b21e89c0550bd11bb849ead0d0c4c3d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:54:15,733 DEBUG [StoreOpener-1b21e89c0550bd11bb849ead0d0c4c3d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d/family2 2023-02-09 09:54:15,734 DEBUG [StoreOpener-1b21e89c0550bd11bb849ead0d0c4c3d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d/family2 2023-02-09 09:54:15,734 INFO [StoreOpener-1b21e89c0550bd11bb849ead0d0c4c3d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b21e89c0550bd11bb849ead0d0c4c3d columnFamilyName family2 2023-02-09 09:54:15,735 INFO [StoreOpener-1b21e89c0550bd11bb849ead0d0c4c3d-1] regionserver.HStore(310): Store=1b21e89c0550bd11bb849ead0d0c4c3d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:15,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:54:15,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:54:15,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:54:15,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:54:15,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:54:15,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1b21e89c0550bd11bb849ead0d0c4c3d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60909287, jitterRate=-0.09238089621067047}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:54:15,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1b21e89c0550bd11bb849ead0d0c4c3d: Running coprocessor pre-open hook at 1675936455725Writing region info on filesystem at 1675936455725Initializing all the Stores at 1675936455726 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936455726Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936455726Cleaning up temporary data from old regions at 1675936455737 (+11 ms)Cleaning up detritus from prior splits at 1675936455738 (+1 ms)Running coprocessor post-open hooks at 1675936455743 (+5 ms)Region opened successfully at 1675936455743 2023-02-09 09:54:15,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d., pid=32, masterSystemTime=1675936455699 2023-02-09 09:54:15,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:15,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:15,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:15,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 086d69d7e75a47257d3c163dd2bc28ef, NAME => 'testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 09:54:15,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:54:15,747 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=1b21e89c0550bd11bb849ead0d0c4c3d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:15,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:15,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:54:15,747 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675936455747"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936455747"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936455747"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936455747"}]},"ts":"1675936455747"} 2023-02-09 09:54:15,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:54:15,748 INFO [StoreOpener-086d69d7e75a47257d3c163dd2bc28ef-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:54:15,750 DEBUG [StoreOpener-086d69d7e75a47257d3c163dd2bc28ef-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef/family1 2023-02-09 09:54:15,750 DEBUG [StoreOpener-086d69d7e75a47257d3c163dd2bc28ef-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef/family1 2023-02-09 09:54:15,751 INFO [StoreOpener-086d69d7e75a47257d3c163dd2bc28ef-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 086d69d7e75a47257d3c163dd2bc28ef columnFamilyName family1 2023-02-09 09:54:15,751 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=29 2023-02-09 09:54:15,751 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=29, state=SUCCESS; OpenRegionProcedure 1b21e89c0550bd11bb849ead0d0c4c3d, server=jenkins-hbase4.apache.org,42719,1675936416061 in 203 msec 2023-02-09 09:54:15,752 INFO [StoreOpener-086d69d7e75a47257d3c163dd2bc28ef-1] regionserver.HStore(310): Store=086d69d7e75a47257d3c163dd2bc28ef/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:15,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1b21e89c0550bd11bb849ead0d0c4c3d, ASSIGN in 367 msec 2023-02-09 09:54:15,752 INFO [StoreOpener-086d69d7e75a47257d3c163dd2bc28ef-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:54:15,754 DEBUG [StoreOpener-086d69d7e75a47257d3c163dd2bc28ef-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef/family2 2023-02-09 09:54:15,754 DEBUG [StoreOpener-086d69d7e75a47257d3c163dd2bc28ef-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef/family2 2023-02-09 09:54:15,754 INFO [StoreOpener-086d69d7e75a47257d3c163dd2bc28ef-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 086d69d7e75a47257d3c163dd2bc28ef columnFamilyName family2 2023-02-09 09:54:15,755 INFO [StoreOpener-086d69d7e75a47257d3c163dd2bc28ef-1] regionserver.HStore(310): Store=086d69d7e75a47257d3c163dd2bc28ef/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:15,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:54:15,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:54:15,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:54:15,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:54:15,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:54:15,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 086d69d7e75a47257d3c163dd2bc28ef; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70000050, jitterRate=0.04308202862739563}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:54:15,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 086d69d7e75a47257d3c163dd2bc28ef: Running coprocessor pre-open hook at 1675936455747Writing region info on filesystem at 1675936455747Initializing all the Stores at 1675936455748 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936455748Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936455748Cleaning up temporary data from old regions at 1675936455758 (+10 ms)Cleaning up detritus from prior splits at 1675936455758Running coprocessor post-open hooks at 1675936455763 (+5 ms)Region opened successfully at 1675936455763 2023-02-09 09:54:15,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef., pid=33, masterSystemTime=1675936455699 2023-02-09 09:54:15,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:15,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:15,766 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=086d69d7e75a47257d3c163dd2bc28ef, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:15,766 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675936455766"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936455766"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936455766"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936455766"}]},"ts":"1675936455766"} 2023-02-09 09:54:15,770 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=28 2023-02-09 09:54:15,770 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=28, state=SUCCESS; OpenRegionProcedure 086d69d7e75a47257d3c163dd2bc28ef, server=jenkins-hbase4.apache.org,42719,1675936416061 in 221 msec 2023-02-09 09:54:15,772 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-02-09 09:54:15,772 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=086d69d7e75a47257d3c163dd2bc28ef, ASSIGN in 386 msec 2023-02-09 09:54:15,773 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:54:15,773 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936455773"}]},"ts":"1675936455773"} 2023-02-09 09:54:15,775 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-02-09 09:54:15,778 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:54:15,780 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2360 sec 2023-02-09 09:54:16,654 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:16,655 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:16,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 09:54:16,656 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-02-09 09:54:16,663 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:16,663 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:16,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:16,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:16,668 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-09 09:54:16,668 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:16,668 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-09 09:54:16,668 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-09 09:54:16,668 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:16,669 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:16,670 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:16,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:16,670 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:16,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:16,670 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:16,671 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:16,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:16,671 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:16,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:16,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:16,671 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:16,672 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:16,672 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-09 09:54:16,672 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:16,672 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-09 09:54:16,672 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:16,672 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. started... 2023-02-09 09:54:16,676 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9e302a2c16508a1b912e987cb0942846 2/2 column families, dataSize=48 B heapSize=632 B 2023-02-09 09:54:16,676 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:16,676 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:54:16,677 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:16,677 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:16,676 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. started... 2023-02-09 09:54:16,677 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. started... 2023-02-09 09:54:16,677 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 1b21e89c0550bd11bb849ead0d0c4c3d: 2023-02-09 09:54:16,678 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 086d69d7e75a47257d3c163dd2bc28ef: 2023-02-09 09:54:16,678 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:16,678 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:16,678 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:54:16,678 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:54:17,092 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209cc997f4e982d4e4bb607686e0893ed34 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209cc997f4e982d4e4bb607686e0893ed34 2023-02-09 09:54:17,498 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/.tmp/family1/23e0ec33bedc4add8e6528d2fd032e82 2023-02-09 09:54:17,505 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/.tmp/family1/23e0ec33bedc4add8e6528d2fd032e82 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/23e0ec33bedc4add8e6528d2fd032e82 2023-02-09 09:54:17,511 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/23e0ec33bedc4add8e6528d2fd032e82, entries=1, sequenceid=5, filesize=4.9 K 2023-02-09 09:54:17,511 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 9e302a2c16508a1b912e987cb0942846 in 835ms, sequenceid=5, compaction requested=false 2023-02-09 09:54:17,512 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-02-09 09:54:17,512 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9e302a2c16508a1b912e987cb0942846: 2023-02-09 09:54:17,512 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:17,512 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:54:17,512 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:54:17,512 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,512 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-09 09:54:17,512 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-09 09:54:17,514 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,514 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:17,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:17,515 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:17,515 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:17,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:17,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:17,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-09 09:54:17,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72bb57d4[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:17,516 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-09 09:54:17,516 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,517 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,518 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-09 09:54:17,518 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-09 09:54:17,518 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:17,518 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,518 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:17,520 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,520 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-09 09:54:17,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,521 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:17,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:17,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:17,521 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-09 09:54:17,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:17,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:17,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:17,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:17,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-09 09:54:17,523 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-09 09:54:17,523 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:17,523 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:17,523 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,523 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:17,525 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,525 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:17,525 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:17,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:17,526 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:17,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,527 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:17,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,531 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,531 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:17,531 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-09 09:54:17,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-09 09:54:17,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:17,531 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:17,531 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:17,531 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:17,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:17,532 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:17,531 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,532 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:17,532 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:17,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:17,532 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,532 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,632 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:17,632 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:17,632 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:17,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:17,638 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:17,638 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:17,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:17,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:17,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,642 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-09 09:54:17,642 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:17,643 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-09 09:54:17,643 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-09 09:54:17,643 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,643 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,644 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,644 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:17,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:17,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:17,645 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:17,645 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:17,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:17,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,646 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-02-09 09:54:17,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:17,648 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-09 09:54:17,648 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:17,648 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-09 09:54:17,653 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:17,653 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. started... 2023-02-09 09:54:17,653 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:17,653 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. started... 2023-02-09 09:54:17,653 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9e302a2c16508a1b912e987cb0942846 2/2 column families, dataSize=90 B heapSize=752 B 2023-02-09 09:54:17,653 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:54:17,653 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 1b21e89c0550bd11bb849ead0d0c4c3d: 2023-02-09 09:54:17,653 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:17,654 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. started... 2023-02-09 09:54:17,653 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:17,654 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 086d69d7e75a47257d3c163dd2bc28ef: 2023-02-09 09:54:17,654 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:54:17,654 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:54:17,654 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:54:17,654 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:54:18,070 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209fe661af84bb945e4be55b6c3eef17a13 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209fe661af84bb945e4be55b6c3eef17a13 2023-02-09 09:54:18,077 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/.tmp/family1/461d665fe8354ad5be83179c2c8fdc08 2023-02-09 09:54:18,084 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/.tmp/family1/461d665fe8354ad5be83179c2c8fdc08 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/461d665fe8354ad5be83179c2c8fdc08 2023-02-09 09:54:18,089 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/461d665fe8354ad5be83179c2c8fdc08, entries=2, sequenceid=9, filesize=5.0 K 2023-02-09 09:54:18,090 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 9e302a2c16508a1b912e987cb0942846 in 437ms, sequenceid=9, compaction requested=false 2023-02-09 09:54:18,091 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9e302a2c16508a1b912e987cb0942846: 2023-02-09 09:54:18,091 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:18,091 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:54:18,091 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:54:18,091 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,091 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-09 09:54:18,091 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-09 09:54:18,093 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,093 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:18,093 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,093 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:18,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:18,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:18,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-09 09:54:18,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@675c19fe[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:18,095 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-09 09:54:18,095 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,096 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-09 09:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-09 09:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:18,097 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,097 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:18,098 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,098 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-09 09:54:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:18,098 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:18,098 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-09 09:54:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:18,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:18,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-09 09:54:18,101 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-09 09:54:18,101 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:18,101 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:18,101 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,101 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:18,105 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,105 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:18,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:18,105 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:18,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:18,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:18,106 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:18,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:18,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,107 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:18,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:18,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,110 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:18,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:18,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-09 09:54:18,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:18,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:18,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:18,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-09 09:54:18,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,110 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:18,111 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:18,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:18,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:18,211 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:18,211 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:18,211 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:18,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:18,219 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-02-09 09:54:18,221 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-09 09:54:18,221 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-09 09:54:18,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:18,222 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937058222 2023-02-09 09:54:18,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:54:18,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-02-09 09:54:18,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 09:54:18,224 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x49e4b84d to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:54:18,231 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@34a90d4d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:54:18,232 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-09 09:54:19,059 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209815daf44c4834ba19df2dd47c05ce709 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209815daf44c4834ba19df2dd47c05ce709 2023-02-09 09:54:19,060 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 09:54:19,073 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:54:19,074 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58912, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:54:19,081 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:19,081 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:19,081 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:54:19,083 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60800, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:54:19,092 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/7673076f2cec4e38b540df72fbbcaab7 first=Optional[k0] last=Optional[k1] 2023-02-09 09:54:19,092 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/7673076f2cec4e38b540df72fbbcaab7}] 2023-02-09 09:54:19,096 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/7673076f2cec4e38b540df72fbbcaab7 for inclusion in 9e302a2c16508a1b912e987cb0942846/family1 2023-02-09 09:54:19,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-02-09 09:54:19,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 09:54:19,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 9e302a2c16508a1b912e987cb0942846: 2023-02-09 09:54:19,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/7673076f2cec4e38b540df72fbbcaab7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testScannerOnBulkLoadRefHFiles__1o2ejcftt529clv6hf5e4u10g4v11hd0o1blbsqe9cqj18scs9d1dp8la88o88s9/family1/7673076f2cec4e38b540df72fbbcaab7 2023-02-09 09:54:19,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testScannerOnBulkLoadRefHFiles__1o2ejcftt529clv6hf5e4u10g4v11hd0o1blbsqe9cqj18scs9d1dp8la88o88s9/family1/7673076f2cec4e38b540df72fbbcaab7 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/1a37c09e3d4c4d80bf0c19d9c99609fd_SeqId_12_ 2023-02-09 09:54:19,103 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testScannerOnBulkLoadRefHFiles__1o2ejcftt529clv6hf5e4u10g4v11hd0o1blbsqe9cqj18scs9d1dp8la88o88s9/family1/7673076f2cec4e38b540df72fbbcaab7 into 9e302a2c16508a1b912e987cb0942846/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/1a37c09e3d4c4d80bf0c19d9c99609fd_SeqId_12_ - updating store file list. 2023-02-09 09:54:19,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/1a37c09e3d4c4d80bf0c19d9c99609fd_SeqId_12_ into 9e302a2c16508a1b912e987cb0942846/family1 2023-02-09 09:54:19,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testScannerOnBulkLoadRefHFiles__1o2ejcftt529clv6hf5e4u10g4v11hd0o1blbsqe9cqj18scs9d1dp8la88o88s9/family1/7673076f2cec4e38b540df72fbbcaab7 into 9e302a2c16508a1b912e987cb0942846/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/1a37c09e3d4c4d80bf0c19d9c99609fd_SeqId_12_) 2023-02-09 09:54:19,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testScannerOnBulkLoadRefHFiles__1o2ejcftt529clv6hf5e4u10g4v11hd0o1blbsqe9cqj18scs9d1dp8la88o88s9/family1/7673076f2cec4e38b540df72fbbcaab7 2023-02-09 09:54:19,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 09:54:19,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 09:54:19,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): 9e302a2c16508a1b912e987cb0942846/family1 is initiating minor compaction (all files) 2023-02-09 09:54:19,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@14627914; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:19,108 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9e302a2c16508a1b912e987cb0942846/family1 in testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:54:19,108 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/23e0ec33bedc4add8e6528d2fd032e82, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/461d665fe8354ad5be83179c2c8fdc08, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/1a37c09e3d4c4d80bf0c19d9c99609fd_SeqId_12_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/.tmp, totalSize=15.1 K 2023-02-09 09:54:19,109 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 23e0ec33bedc4add8e6528d2fd032e82, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936456656 2023-02-09 09:54:19,109 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 461d665fe8354ad5be83179c2c8fdc08, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936456656 2023-02-09 09:54:19,110 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 1a37c09e3d4c4d80bf0c19d9c99609fd_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936456656 2023-02-09 09:54:19,112 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 09:54:19,113 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:19,115 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209cc997f4e982d4e4bb607686e0893ed34 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209cc997f4e982d4e4bb607686e0893ed34 2023-02-09 09:54:19,116 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209fe661af84bb945e4be55b6c3eef17a13 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209fe661af84bb945e4be55b6c3eef17a13 2023-02-09 09:54:19,116 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-02-09 09:54:19,116 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 09:54:19,117 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-09 09:54:19,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 09:54:19,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:19,118 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-09 09:54:19,119 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 898 msec 2023-02-09 09:54:19,127 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9e302a2c16508a1b912e987cb0942846#family1#compaction#37 average throughput is 0.02 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:54:19,151 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=251 (was 256), OpenFileDescriptor=605 (was 619), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=359 (was 303) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3529 (was 3626) 2023-02-09 09:54:19,161 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=251, OpenFileDescriptor=605, MaxFileDescriptor=60000, SystemLoadAverage=359, ProcessCount=168, AvailableMemoryMB=3528 2023-02-09 09:54:19,162 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:19,162 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:19,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 09:54:19,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:19,166 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:54:19,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-02-09 09:54:19,166 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:19,166 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:19,166 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:54:19,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 09:54:19,168 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44 2023-02-09 09:54:19,168 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44 empty. 2023-02-09 09:54:19,169 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44 2023-02-09 09:54:19,169 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-02-09 09:54:19,267 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:19,267 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:19,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 09:54:19,468 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:19,469 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:19,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 09:54:19,577 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 09:54:19,579 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 651cd17c19c5ed7af78ba9deadfd9f44, NAME => 'testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:54:19,770 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:19,770 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:19,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 09:54:19,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:19,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 651cd17c19c5ed7af78ba9deadfd9f44, disabling compactions & flushes 2023-02-09 09:54:19,985 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:19,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:19,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. after waiting 0 ms 2023-02-09 09:54:19,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:19,985 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:19,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 651cd17c19c5ed7af78ba9deadfd9f44: Waiting for close lock at 1675936459985Disabling compacts and flushes for region at 1675936459985Disabling writes for close at 1675936459985Writing region close event to WAL at 1675936459985Closed at 1675936459985 2023-02-09 09:54:19,988 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:54:19,989 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675936459989"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936459989"}]},"ts":"1675936459989"} 2023-02-09 09:54:19,990 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 09:54:19,991 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:54:19,991 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936459991"}]},"ts":"1675936459991"} 2023-02-09 09:54:19,992 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-09 09:54:19,996 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=651cd17c19c5ed7af78ba9deadfd9f44, ASSIGN}] 2023-02-09 09:54:19,997 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=651cd17c19c5ed7af78ba9deadfd9f44, ASSIGN 2023-02-09 09:54:19,998 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=651cd17c19c5ed7af78ba9deadfd9f44, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:54:20,149 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=651cd17c19c5ed7af78ba9deadfd9f44, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:20,149 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675936460149"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936460149"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936460149"}]},"ts":"1675936460149"} 2023-02-09 09:54:20,151 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 651cd17c19c5ed7af78ba9deadfd9f44, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:20,271 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:20,271 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:20,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 09:54:20,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:20,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 651cd17c19c5ed7af78ba9deadfd9f44, NAME => 'testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:54:20,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 651cd17c19c5ed7af78ba9deadfd9f44 2023-02-09 09:54:20,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:20,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 651cd17c19c5ed7af78ba9deadfd9f44 2023-02-09 09:54:20,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 651cd17c19c5ed7af78ba9deadfd9f44 2023-02-09 09:54:20,307 INFO [StoreOpener-651cd17c19c5ed7af78ba9deadfd9f44-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 651cd17c19c5ed7af78ba9deadfd9f44 2023-02-09 09:54:20,308 DEBUG [StoreOpener-651cd17c19c5ed7af78ba9deadfd9f44-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1 2023-02-09 09:54:20,308 DEBUG [StoreOpener-651cd17c19c5ed7af78ba9deadfd9f44-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1 2023-02-09 09:54:20,309 INFO [StoreOpener-651cd17c19c5ed7af78ba9deadfd9f44-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 651cd17c19c5ed7af78ba9deadfd9f44 columnFamilyName family1 2023-02-09 09:54:20,310 INFO [StoreOpener-651cd17c19c5ed7af78ba9deadfd9f44-1] regionserver.HStore(310): Store=651cd17c19c5ed7af78ba9deadfd9f44/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:20,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44 2023-02-09 09:54:20,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44 2023-02-09 09:54:20,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 651cd17c19c5ed7af78ba9deadfd9f44 2023-02-09 09:54:20,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:54:20,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 651cd17c19c5ed7af78ba9deadfd9f44; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60685774, jitterRate=-0.09571149945259094}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:54:20,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 651cd17c19c5ed7af78ba9deadfd9f44: Running coprocessor pre-open hook at 1675936460306Writing region info on filesystem at 1675936460306Initializing all the Stores at 1675936460307 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675936460307Cleaning up temporary data from old regions at 1675936460312 (+5 ms)Cleaning up detritus from prior splits at 1675936460312Running coprocessor post-open hooks at 1675936460316 (+4 ms)Region opened successfully at 1675936460316 2023-02-09 09:54:20,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44., pid=37, masterSystemTime=1675936460302 2023-02-09 09:54:20,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:20,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:20,319 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=651cd17c19c5ed7af78ba9deadfd9f44, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:20,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675936460319"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936460319"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936460319"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936460319"}]},"ts":"1675936460319"} 2023-02-09 09:54:20,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-02-09 09:54:20,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 651cd17c19c5ed7af78ba9deadfd9f44, server=jenkins-hbase4.apache.org,42719,1675936416061 in 169 msec 2023-02-09 09:54:20,324 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-09 09:54:20,324 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=651cd17c19c5ed7af78ba9deadfd9f44, ASSIGN in 326 msec 2023-02-09 09:54:20,324 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:54:20,324 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936460324"}]},"ts":"1675936460324"} 2023-02-09 09:54:20,325 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-09 09:54:20,327 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:54:20,328 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1640 sec 2023-02-09 09:54:20,347 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/.tmp/family1/b7bc488526214db3bf5cb6c603b2c05a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/b7bc488526214db3bf5cb6c603b2c05a 2023-02-09 09:54:20,353 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 9e302a2c16508a1b912e987cb0942846/family1 of 9e302a2c16508a1b912e987cb0942846 into b7bc488526214db3bf5cb6c603b2c05a(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:54:20,353 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9e302a2c16508a1b912e987cb0942846: 2023-02-09 09:54:20,353 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1675936459108; duration=1sec 2023-02-09 09:54:20,353 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:20,430 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/23e0ec33bedc4add8e6528d2fd032e82, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/461d665fe8354ad5be83179c2c8fdc08, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/1a37c09e3d4c4d80bf0c19d9c99609fd_SeqId_12_] to archive 2023-02-09 09:54:20,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:20,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/23e0ec33bedc4add8e6528d2fd032e82 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/23e0ec33bedc4add8e6528d2fd032e82 2023-02-09 09:54:20,434 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/461d665fe8354ad5be83179c2c8fdc08 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/461d665fe8354ad5be83179c2c8fdc08 2023-02-09 09:54:20,435 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/1a37c09e3d4c4d80bf0c19d9c99609fd_SeqId_12_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/family1/1a37c09e3d4c4d80bf0c19d9c99609fd_SeqId_12_ 2023-02-09 09:54:21,205 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:54:21,272 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:21,273 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:21,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 09:54:21,274 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-02-09 09:54:21,295 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:21,295 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:21,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:21,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:21,297 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:21,297 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:21,298 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:21,298 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:21,298 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:21,298 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:21,299 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:21,299 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:21,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:21,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:21,300 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:21,300 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:21,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:21,300 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:21,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:21,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:21,300 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:21,301 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:21,301 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:21,301 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:21,301 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:21,301 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:21,301 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:21,301 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:21,301 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. started... 2023-02-09 09:54:21,302 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 651cd17c19c5ed7af78ba9deadfd9f44 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 09:54:21,721 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130986149e24c3e4207a2e2097856796ade to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130986149e24c3e4207a2e2097856796ade 2023-02-09 09:54:22,125 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/26badea380234f70b06d65eb90830cfd 2023-02-09 09:54:22,131 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/26badea380234f70b06d65eb90830cfd as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/26badea380234f70b06d65eb90830cfd 2023-02-09 09:54:22,135 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/26badea380234f70b06d65eb90830cfd, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-09 09:54:22,136 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 651cd17c19c5ed7af78ba9deadfd9f44 in 834ms, sequenceid=5, compaction requested=false 2023-02-09 09:54:22,136 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:22,136 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:22,136 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:22,136 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:22,137 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:22,137 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,137 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:22,137 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:22,138 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,138 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:22,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:22,138 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,139 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:22,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:22,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:22,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:22,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:22,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74e71347[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:22,140 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:22,140 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,142 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,142 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:22,142 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:22,142 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:22,142 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,143 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:22,144 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:22,144 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,144 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:22,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:22,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:22,144 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:22,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:22,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:22,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:22,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:22,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:22,146 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:22,146 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:22,146 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:22,146 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,146 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:22,148 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,148 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:22,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,148 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:22,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:22,149 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:22,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,150 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,152 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:22,152 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:22,152 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:22,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:22,152 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,152 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:22,152 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:22,152 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:22,152 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,152 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,152 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,152 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,152 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:22,153 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:22,153 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:22,253 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:22,253 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:22,253 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:22,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:22,258 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:22,258 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:22,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:22,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:22,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,260 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:22,260 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:22,260 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:22,260 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:22,261 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,261 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,263 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,263 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:22,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:22,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:22,264 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:22,264 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:22,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:22,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,264 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,265 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:22,265 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:22,265 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:22,265 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:22,265 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:22,265 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:22,265 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:22,265 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:22,265 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. started... 2023-02-09 09:54:22,265 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 651cd17c19c5ed7af78ba9deadfd9f44 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 09:54:22,679 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b8ddfbdaefa949ba884d5bcb7a19eceb to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128b8ddfbdaefa949ba884d5bcb7a19eceb 2023-02-09 09:54:23,084 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/301d8d42bac74bb684b240f63679a2cd 2023-02-09 09:54:23,090 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/301d8d42bac74bb684b240f63679a2cd as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/301d8d42bac74bb684b240f63679a2cd 2023-02-09 09:54:23,094 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/301d8d42bac74bb684b240f63679a2cd, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 09:54:23,095 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 651cd17c19c5ed7af78ba9deadfd9f44 in 830ms, sequenceid=9, compaction requested=false 2023-02-09 09:54:23,095 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:23,095 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:23,095 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:23,095 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:23,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:23,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:23,097 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,097 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:23,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:23,098 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,098 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:23,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:23,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:23,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:23,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:23,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46254afd[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:23,099 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:23,099 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,100 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,100 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:23,100 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:23,100 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:23,100 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,100 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:23,106 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:23,106 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,106 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:23,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:23,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:23,106 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:23,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:23,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:23,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:23,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:23,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:23,108 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:23,108 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:23,108 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:23,108 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,108 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:23,109 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,109 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:23,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:23,109 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:23,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:23,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:23,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:23,110 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:23,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,111 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:23,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,113 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,113 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:23,113 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:23,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:23,113 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:23,113 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:23,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:23,113 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:23,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:23,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:23,113 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,114 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:23,114 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:23,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,214 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:23,214 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:23,214 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:23,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:23,219 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:23,219 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:23,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:23,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:23,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,221 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:23,221 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:23,221 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:23,222 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:23,222 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,222 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,224 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:23,224 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:23,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:23,224 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:23,224 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:23,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:23,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,225 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:23,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:23,225 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:23,225 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:23,225 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:23,226 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:23,226 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:23,226 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:23,226 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. started... 2023-02-09 09:54:23,226 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 651cd17c19c5ed7af78ba9deadfd9f44 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:23,639 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120104aba48088714903bdf3d3a439a47920 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120104aba48088714903bdf3d3a439a47920 2023-02-09 09:54:24,044 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/46d78cf3dabb4802a9efe2f32163e1b8 2023-02-09 09:54:24,049 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/46d78cf3dabb4802a9efe2f32163e1b8 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/46d78cf3dabb4802a9efe2f32163e1b8 2023-02-09 09:54:24,054 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/46d78cf3dabb4802a9efe2f32163e1b8, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 09:54:24,055 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 651cd17c19c5ed7af78ba9deadfd9f44 in 829ms, sequenceid=13, compaction requested=true 2023-02-09 09:54:24,055 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:24,055 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:24,055 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:24,055 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:24,055 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,055 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:24,055 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:24,057 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,057 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:24,058 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,058 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:24,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:24,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e08b01a[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:24,059 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:24,059 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,060 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,061 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:24,061 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:24,061 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:24,061 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,061 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:24,062 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:24,062 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:24,062 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,062 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:24,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:24,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:24,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:24,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:24,064 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:24,064 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:24,064 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:24,064 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,064 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:24,066 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,066 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:24,066 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:24,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:24,066 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,067 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:24,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,070 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,070 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,070 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:24,070 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:24,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:24,070 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:24,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:24,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:24,070 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:24,071 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,071 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,071 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,071 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:24,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:24,071 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:24,071 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:24,171 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:24,171 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:24,171 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:24,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:24,176 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:24,176 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:24,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:24,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:24,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,178 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:24,178 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:24,179 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:24,179 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:24,179 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,179 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,180 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,180 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:24,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:24,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:24,180 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:24,180 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:24,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:24,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,181 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,181 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:24,181 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:24,181 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:24,181 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:24,181 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:24,182 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:24,182 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:24,182 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:24,182 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. started... 2023-02-09 09:54:24,182 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 651cd17c19c5ed7af78ba9deadfd9f44 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:24,597 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120547b39ea1d3b34504a11a3eb6d80dcfa7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120547b39ea1d3b34504a11a3eb6d80dcfa7 2023-02-09 09:54:25,003 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/92b633530806405fbef90600caac3aca 2023-02-09 09:54:25,008 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/92b633530806405fbef90600caac3aca as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b633530806405fbef90600caac3aca 2023-02-09 09:54:25,013 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b633530806405fbef90600caac3aca, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 09:54:25,014 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 651cd17c19c5ed7af78ba9deadfd9f44 in 832ms, sequenceid=17, compaction requested=true 2023-02-09 09:54:25,014 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:25,014 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:25,014 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:25,014 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:25,014 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,014 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:25,014 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:25,016 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,016 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:25,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:25,016 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,016 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:25,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:25,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:25,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:25,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:25,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@699cc7cc[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:25,018 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:25,018 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,019 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,020 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:25,020 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:25,020 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:25,020 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,020 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:25,022 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:25,022 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,022 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:25,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:25,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:25,022 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:25,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:25,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:25,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:25,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:25,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:25,025 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:25,025 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:25,025 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:25,025 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,025 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:25,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:25,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:25,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:25,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:25,027 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:25,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:25,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:25,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,029 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:25,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,031 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,031 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:25,031 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:25,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:25,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:25,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:25,031 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,031 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:25,031 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:25,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:25,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:25,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:25,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,032 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:25,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,032 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:25,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,132 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:25,133 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:25,133 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:25,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:25,137 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:25,137 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:25,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:25,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:25,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,139 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:25,139 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:25,139 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:25,139 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:25,140 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,140 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,141 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:25,141 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:25,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:25,141 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,141 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:25,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,142 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:25,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,142 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:25,143 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,143 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:25,143 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:25,143 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:25,143 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:25,143 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:25,143 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:25,143 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. started... 2023-02-09 09:54:25,143 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 651cd17c19c5ed7af78ba9deadfd9f44 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:25,557 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512287677f29c427746dcaa96d2c0d7285320 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512287677f29c427746dcaa96d2c0d7285320 2023-02-09 09:54:25,963 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/9ec84a3a5f7443d0ab466dc84dc6cd76 2023-02-09 09:54:25,968 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/9ec84a3a5f7443d0ab466dc84dc6cd76 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/9ec84a3a5f7443d0ab466dc84dc6cd76 2023-02-09 09:54:25,972 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/9ec84a3a5f7443d0ab466dc84dc6cd76, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 09:54:25,972 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 651cd17c19c5ed7af78ba9deadfd9f44 in 829ms, sequenceid=21, compaction requested=true 2023-02-09 09:54:25,972 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:25,972 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:25,973 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:25,973 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:25,973 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,973 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:25,973 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:25,978 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,978 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:25,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:25,978 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,978 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:25,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:25,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:25,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:25,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:25,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25ad9c93[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:25,980 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:25,980 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,981 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,981 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:25,981 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:25,981 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:25,981 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,981 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:25,982 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:25,982 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:25,982 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,982 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:25,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:25,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:25,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:25,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:25,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:25,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:25,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:25,985 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:25,985 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:25,985 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:25,985 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,985 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:25,987 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,987 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:25,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:25,987 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:25,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:25,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:25,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:25,987 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:25,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,988 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:25,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,991 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,991 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:25,991 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:25,992 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:25,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:25,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:25,992 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,991 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:25,992 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:25,992 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,992 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:25,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:25,992 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:25,992 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:25,992 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,092 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:26,092 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:26,093 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:26,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:26,097 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:26,097 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:26,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:26,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:26,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,099 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:26,099 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:26,099 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:26,099 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:26,099 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,099 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,100 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:26,100 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:26,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:26,101 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,101 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:26,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,101 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:26,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,101 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,102 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:26,102 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:26,102 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:26,102 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. started... 2023-02-09 09:54:26,102 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 651cd17c19c5ed7af78ba9deadfd9f44 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:26,515 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231cb5ee238869141058568f63c32fba81c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231cb5ee238869141058568f63c32fba81c 2023-02-09 09:54:26,920 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/588b757e9395467f93fda2bfed3ba87e 2023-02-09 09:54:26,924 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/588b757e9395467f93fda2bfed3ba87e as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/588b757e9395467f93fda2bfed3ba87e 2023-02-09 09:54:26,928 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/588b757e9395467f93fda2bfed3ba87e, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 09:54:26,929 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 651cd17c19c5ed7af78ba9deadfd9f44 in 827ms, sequenceid=25, compaction requested=true 2023-02-09 09:54:26,929 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:26,929 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:26,929 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:26,929 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:26,929 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,929 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:26,929 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:26,931 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,931 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:26,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:26,931 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,931 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:26,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:26,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:26,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:26,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:26,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49b34879[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:26,932 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:26,933 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,934 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:26,934 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,934 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:26,935 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:26,935 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,935 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:26,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:26,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:26,935 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:26,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:26,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:26,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:26,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:26,938 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:26,938 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:26,938 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:26,938 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:26,938 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,938 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:26,940 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,940 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,940 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:26,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:26,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:26,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:26,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:26,940 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:26,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:26,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,941 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:26,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,943 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,943 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:26,943 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:26,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:26,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:26,943 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,943 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:26,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:26,943 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:26,943 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:26,944 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:26,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:26,944 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:26,944 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:26,944 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:26,944 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:26,944 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,044 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:27,044 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:27,045 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:27,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:27,049 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:27,049 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:27,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:27,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:27,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,052 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:27,052 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:27,052 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:27,052 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:27,052 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,052 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,054 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,054 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:27,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:27,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:27,054 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,054 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:27,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:27,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,056 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,056 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:27,056 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,056 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:27,056 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:27,056 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:27,056 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:27,056 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:27,056 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:27,056 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. started... 2023-02-09 09:54:27,056 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 651cd17c19c5ed7af78ba9deadfd9f44 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:27,471 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601012f06e85eb9784035a1425aee1bd1ded3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601012f06e85eb9784035a1425aee1bd1ded3 2023-02-09 09:54:27,877 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/543d73a626a94639aefd049616fe0318 2023-02-09 09:54:27,881 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/543d73a626a94639aefd049616fe0318 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/543d73a626a94639aefd049616fe0318 2023-02-09 09:54:27,885 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/543d73a626a94639aefd049616fe0318, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 09:54:27,886 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 651cd17c19c5ed7af78ba9deadfd9f44 in 830ms, sequenceid=29, compaction requested=true 2023-02-09 09:54:27,886 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:27,886 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:27,886 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:27,886 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:27,886 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,886 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:27,886 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:27,888 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,888 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:27,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:27,888 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,888 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:27,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:27,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:27,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:27,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:27,890 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6210a935[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:27,890 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:27,890 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,891 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,891 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:27,891 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:27,891 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:27,891 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,891 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:27,893 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:27,893 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,893 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:27,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:27,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:27,893 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:27,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:27,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:27,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:27,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:27,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:27,895 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:27,895 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:27,895 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:27,895 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,895 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:27,897 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,897 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:27,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:27,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,897 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:27,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:27,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:27,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:27,897 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:27,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,898 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:27,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,900 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,900 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:27,900 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,900 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:27,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:27,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:27,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:27,900 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,900 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:27,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:27,901 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:27,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:27,901 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,901 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:27,901 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:27,901 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,901 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:27,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:27,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:28,001 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:28,001 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:28,001 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:28,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:28,007 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:28,007 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:28,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:28,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:28,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,009 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:28,009 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:28,009 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:28,009 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:28,010 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,010 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,014 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:28,014 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:28,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:28,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,015 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,015 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:28,016 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:28,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,016 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,016 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:28,017 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,017 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:28,017 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:28,017 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:28,017 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:28,017 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:28,017 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:28,017 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. started... 2023-02-09 09:54:28,017 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 651cd17c19c5ed7af78ba9deadfd9f44 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:28,434 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032866386440f243bd9449a6dcd145c999 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601032866386440f243bd9449a6dcd145c999 2023-02-09 09:54:28,840 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/ab838828936d435a98402459a5247a15 2023-02-09 09:54:28,845 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/ab838828936d435a98402459a5247a15 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/ab838828936d435a98402459a5247a15 2023-02-09 09:54:28,849 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/ab838828936d435a98402459a5247a15, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 09:54:28,850 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 651cd17c19c5ed7af78ba9deadfd9f44 in 833ms, sequenceid=33, compaction requested=true 2023-02-09 09:54:28,850 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:28,850 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:28,850 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:28,851 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:28,851 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,851 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:28,851 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:28,852 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,852 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:28,852 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:28,852 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d3da01c[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:28,854 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:28,854 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,855 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,855 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:28,855 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:28,855 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:28,855 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,855 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:28,856 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:28,856 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,856 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:28,856 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:28,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:28,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:28,859 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:28,859 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:28,859 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:28,859 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,859 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:28,861 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,861 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:28,861 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:28,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:28,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:28,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:28,862 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:28,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:28,866 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:28,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,869 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:28,869 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,869 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:28,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:28,869 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:28,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:28,869 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:28,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:28,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:28,869 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,869 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:28,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:28,869 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,869 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,869 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,869 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:28,870 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:28,970 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:28,970 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:28,970 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:28,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:28,974 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:28,974 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:28,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:28,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:28,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,976 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:28,976 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:28,976 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:28,976 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:28,977 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,977 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,978 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:28,978 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:28,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:28,978 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:28,978 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:28,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,978 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:28,979 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:28,979 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:28,979 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:28,979 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:28,979 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:28,979 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:28,980 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. started... 2023-02-09 09:54:28,980 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 651cd17c19c5ed7af78ba9deadfd9f44 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:29,393 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078aea3954d9e44ae783778220194f8c8b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201509078aea3954d9e44ae783778220194f8c8b 2023-02-09 09:54:29,399 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/a92f70dca41d459680717060df8c201a 2023-02-09 09:54:29,403 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/a92f70dca41d459680717060df8c201a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/a92f70dca41d459680717060df8c201a 2023-02-09 09:54:29,408 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/a92f70dca41d459680717060df8c201a, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 09:54:29,408 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 651cd17c19c5ed7af78ba9deadfd9f44 in 428ms, sequenceid=37, compaction requested=true 2023-02-09 09:54:29,408 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:29,409 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:29,409 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:29,409 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:29,409 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,409 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:29,409 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:29,410 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,410 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:29,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:29,411 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,411 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:29,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:29,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:29,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:29,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:29,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@243df02c[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:29,412 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:29,412 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,415 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,415 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:29,415 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:29,415 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:29,415 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,415 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:29,416 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,416 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:29,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:29,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:29,416 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:29,416 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:29,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:29,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:29,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:29,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:29,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:29,419 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:29,419 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:29,419 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:29,419 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,419 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:29,420 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,420 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:29,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:29,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,420 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:29,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:29,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:29,421 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:29,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,422 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,424 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,424 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,424 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:29,424 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,424 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:29,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:29,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:29,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:29,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:29,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:29,424 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:29,424 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,425 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:29,425 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:29,425 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:29,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:29,425 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,425 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,525 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:29,525 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:29,525 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:29,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:29,530 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:29,530 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:29,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:29,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:29,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,532 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:29,532 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:29,532 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 09:54:29,532 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 09:54:29,532 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,532 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,534 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:29,534 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:29,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:29,535 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:29,535 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:29,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,535 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:29,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,535 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,536 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 09:54:29,536 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:29,536 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 09:54:29,536 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:29,536 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 09:54:29,536 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:29,536 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:29,536 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:29,536 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. started... 2023-02-09 09:54:29,536 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 651cd17c19c5ed7af78ba9deadfd9f44 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:29,949 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e4ce0faf090a492b9df5303d5cf4d834 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120e4ce0faf090a492b9df5303d5cf4d834 2023-02-09 09:54:30,356 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/5096e3e103244604bdffd85db9aca2e5 2023-02-09 09:54:30,361 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/5096e3e103244604bdffd85db9aca2e5 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/5096e3e103244604bdffd85db9aca2e5 2023-02-09 09:54:30,366 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/5096e3e103244604bdffd85db9aca2e5, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 09:54:30,367 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 651cd17c19c5ed7af78ba9deadfd9f44 in 831ms, sequenceid=41, compaction requested=true 2023-02-09 09:54:30,367 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:30,367 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 09:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 09:54:30,369 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,369 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:30,369 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,369 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:30,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 09:54:30,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@771e9452[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:30,371 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 09:54:30,371 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,372 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,372 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 09:54:30,372 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 09:54:30,372 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:30,372 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,372 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:30,373 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 09:54:30,373 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,373 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:30,374 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 09:54:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:30,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:54:30,376 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 09:54:30,376 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:30,376 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:30,376 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,376 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:30,379 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,379 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:30,379 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:30,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:30,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:30,380 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:30,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:30,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:30,381 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:30,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,387 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,387 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,387 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:30,387 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:30,387 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:30,387 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:30,387 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,387 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:30,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 09:54:30,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:30,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:30,387 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,388 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 09:54:30,388 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:30,388 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:30,488 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:30,488 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:30,488 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:30,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:30,490 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-02-09 09:54:30,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:54:30,493 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:54:30,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:30,494 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937070494 2023-02-09 09:54:30,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:54:30,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-09 09:54:30,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:30,499 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x1f9e2030 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:54:30,507 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7c41a17, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:54:30,507 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:31,356 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035091b0746a58421884b6687f4e84d24d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601035091b0746a58421884b6687f4e84d24d 2023-02-09 09:54:31,357 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:31,381 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:54:31,381 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x49e4b84d to 127.0.0.1:64170 2023-02-09 09:54:31,381 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:54:31,381 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:54:31,381 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x68a0246a to 127.0.0.1:64170 2023-02-09 09:54:31,381 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:54:31,381 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:54:31,381 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x757dee88 to 127.0.0.1:64170 2023-02-09 09:54:31,381 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:54:31,388 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:54:31,389 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40072, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:54:31,395 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:31,395 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:31,395 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:54:31,396 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48298, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:54:31,404 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ee7fd23c0c204706a960a2ede4006c5f first=Optional[r3] last=Optional[r6] 2023-02-09 09:54:31,404 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ee7fd23c0c204706a960a2ede4006c5f}] 2023-02-09 09:54:31,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ee7fd23c0c204706a960a2ede4006c5f for inclusion in 651cd17c19c5ed7af78ba9deadfd9f44/family1 2023-02-09 09:54:31,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 09:54:31,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:54:31,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:31,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ee7fd23c0c204706a960a2ede4006c5f to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__a9sejjaep59q07o8igs22qhj78spllg646km86niq7cc9k4n654fvebhes7gkh51/family1/ee7fd23c0c204706a960a2ede4006c5f 2023-02-09 09:54:31,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__a9sejjaep59q07o8igs22qhj78spllg646km86niq7cc9k4n654fvebhes7gkh51/family1/ee7fd23c0c204706a960a2ede4006c5f as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/d397cb16a351444991784cee13a9ef60_SeqId_44_ 2023-02-09 09:54:31,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__a9sejjaep59q07o8igs22qhj78spllg646km86niq7cc9k4n654fvebhes7gkh51/family1/ee7fd23c0c204706a960a2ede4006c5f into 651cd17c19c5ed7af78ba9deadfd9f44/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/d397cb16a351444991784cee13a9ef60_SeqId_44_ - updating store file list. 2023-02-09 09:54:31,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/d397cb16a351444991784cee13a9ef60_SeqId_44_ into 651cd17c19c5ed7af78ba9deadfd9f44/family1 2023-02-09 09:54:31,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__a9sejjaep59q07o8igs22qhj78spllg646km86niq7cc9k4n654fvebhes7gkh51/family1/ee7fd23c0c204706a960a2ede4006c5f into 651cd17c19c5ed7af78ba9deadfd9f44/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/d397cb16a351444991784cee13a9ef60_SeqId_44_) 2023-02-09 09:54:31,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__a9sejjaep59q07o8igs22qhj78spllg646km86niq7cc9k4n654fvebhes7gkh51/family1/ee7fd23c0c204706a960a2ede4006c5f 2023-02-09 09:54:31,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 09:54:31,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 09:54:31,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): 651cd17c19c5ed7af78ba9deadfd9f44/family1 is initiating minor compaction 2023-02-09 09:54:31,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@72479556; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:31,424 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 651cd17c19c5ed7af78ba9deadfd9f44/family1 in testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:31,424 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/301d8d42bac74bb684b240f63679a2cd, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/46d78cf3dabb4802a9efe2f32163e1b8, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b633530806405fbef90600caac3aca, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/9ec84a3a5f7443d0ab466dc84dc6cd76, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/588b757e9395467f93fda2bfed3ba87e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/543d73a626a94639aefd049616fe0318, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/ab838828936d435a98402459a5247a15, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/a92f70dca41d459680717060df8c201a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/5096e3e103244604bdffd85db9aca2e5, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/d397cb16a351444991784cee13a9ef60_SeqId_44_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp, totalSize=50.0 K 2023-02-09 09:54:31,424 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 301d8d42bac74bb684b240f63679a2cd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 09:54:31,424 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 46d78cf3dabb4802a9efe2f32163e1b8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 09:54:31,424 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 92b633530806405fbef90600caac3aca, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 09:54:31,424 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 9ec84a3a5f7443d0ab466dc84dc6cd76, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 09:54:31,425 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 588b757e9395467f93fda2bfed3ba87e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 09:54:31,425 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 543d73a626a94639aefd049616fe0318, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 09:54:31,425 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting ab838828936d435a98402459a5247a15, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 09:54:31,425 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting a92f70dca41d459680717060df8c201a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 09:54:31,425 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 5096e3e103244604bdffd85db9aca2e5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 09:54:31,425 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting d397cb16a351444991784cee13a9ef60_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 09:54:31,426 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:31,426 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:31,428 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512287677f29c427746dcaa96d2c0d7285320 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512287677f29c427746dcaa96d2c0d7285320 2023-02-09 09:54:31,429 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231cb5ee238869141058568f63c32fba81c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231cb5ee238869141058568f63c32fba81c 2023-02-09 09:54:31,430 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601012f06e85eb9784035a1425aee1bd1ded3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601012f06e85eb9784035a1425aee1bd1ded3 2023-02-09 09:54:31,431 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601032866386440f243bd9449a6dcd145c999 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601032866386440f243bd9449a6dcd145c999 2023-02-09 09:54:31,432 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-09 09:54:31,432 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 09:54:31,432 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:31,432 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:31,432 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:31,432 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:31,432 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:31,432 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:31,432 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:31,433 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:31,459 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 651cd17c19c5ed7af78ba9deadfd9f44#family1#compaction#48 average throughput is 1.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:54:32,276 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/92b6aa2c79884998bce4baa6427d3ce7 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b6aa2c79884998bce4baa6427d3ce7 2023-02-09 09:54:32,281 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 651cd17c19c5ed7af78ba9deadfd9f44/family1 of 651cd17c19c5ed7af78ba9deadfd9f44 into 92b6aa2c79884998bce4baa6427d3ce7(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:54:32,281 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:32,281 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1675936471424; duration=0sec 2023-02-09 09:54:32,282 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:32,299 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205bd36147b16fc40c1a4250c31e70a0a0e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205bd36147b16fc40c1a4250c31e70a0a0e 2023-02-09 09:54:32,300 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:32,316 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:32,317 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:32,324 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d630f9658669418b9b67bf21b49fae5b first=Optional[r00] last=Optional[r2] 2023-02-09 09:54:32,325 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d630f9658669418b9b67bf21b49fae5b}] 2023-02-09 09:54:32,327 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d630f9658669418b9b67bf21b49fae5b for inclusion in 651cd17c19c5ed7af78ba9deadfd9f44/family1 2023-02-09 09:54:32,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/301d8d42bac74bb684b240f63679a2cd, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/46d78cf3dabb4802a9efe2f32163e1b8, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b633530806405fbef90600caac3aca, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/9ec84a3a5f7443d0ab466dc84dc6cd76, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/588b757e9395467f93fda2bfed3ba87e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/543d73a626a94639aefd049616fe0318, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/ab838828936d435a98402459a5247a15, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/a92f70dca41d459680717060df8c201a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/5096e3e103244604bdffd85db9aca2e5, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/d397cb16a351444991784cee13a9ef60_SeqId_44_] to archive 2023-02-09 09:54:32,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:32,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 09:54:32,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:54:32,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:32,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d630f9658669418b9b67bf21b49fae5b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__34cia8qfmn6et506dt9um87r2ig4i6d2h8cn2jvhmkbej543u2ed91h8ie28q3pq/family1/d630f9658669418b9b67bf21b49fae5b 2023-02-09 09:54:32,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/301d8d42bac74bb684b240f63679a2cd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/301d8d42bac74bb684b240f63679a2cd 2023-02-09 09:54:32,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__34cia8qfmn6et506dt9um87r2ig4i6d2h8cn2jvhmkbej543u2ed91h8ie28q3pq/family1/d630f9658669418b9b67bf21b49fae5b as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/7df0293ab53049bbb865acbe7db749fe_SeqId_47_ 2023-02-09 09:54:32,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/46d78cf3dabb4802a9efe2f32163e1b8 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/46d78cf3dabb4802a9efe2f32163e1b8 2023-02-09 09:54:32,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__34cia8qfmn6et506dt9um87r2ig4i6d2h8cn2jvhmkbej543u2ed91h8ie28q3pq/family1/d630f9658669418b9b67bf21b49fae5b into 651cd17c19c5ed7af78ba9deadfd9f44/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/7df0293ab53049bbb865acbe7db749fe_SeqId_47_ - updating store file list. 2023-02-09 09:54:32,346 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b633530806405fbef90600caac3aca to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b633530806405fbef90600caac3aca 2023-02-09 09:54:32,347 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/9ec84a3a5f7443d0ab466dc84dc6cd76 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/9ec84a3a5f7443d0ab466dc84dc6cd76 2023-02-09 09:54:32,348 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/588b757e9395467f93fda2bfed3ba87e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/588b757e9395467f93fda2bfed3ba87e 2023-02-09 09:54:32,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/7df0293ab53049bbb865acbe7db749fe_SeqId_47_ into 651cd17c19c5ed7af78ba9deadfd9f44/family1 2023-02-09 09:54:32,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__34cia8qfmn6et506dt9um87r2ig4i6d2h8cn2jvhmkbej543u2ed91h8ie28q3pq/family1/d630f9658669418b9b67bf21b49fae5b into 651cd17c19c5ed7af78ba9deadfd9f44/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/7df0293ab53049bbb865acbe7db749fe_SeqId_47_) 2023-02-09 09:54:32,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyPolicy__34cia8qfmn6et506dt9um87r2ig4i6d2h8cn2jvhmkbej543u2ed91h8ie28q3pq/family1/d630f9658669418b9b67bf21b49fae5b 2023-02-09 09:54:32,349 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/543d73a626a94639aefd049616fe0318 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/543d73a626a94639aefd049616fe0318 2023-02-09 09:54:32,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 09:54:32,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 09:54:32,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): 651cd17c19c5ed7af78ba9deadfd9f44/family1 is initiating minor compaction (all files) 2023-02-09 09:54:32,351 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/ab838828936d435a98402459a5247a15 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/ab838828936d435a98402459a5247a15 2023-02-09 09:54:32,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@262adb6b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:32,351 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 651cd17c19c5ed7af78ba9deadfd9f44/family1 in testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:54:32,351 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/26badea380234f70b06d65eb90830cfd, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b6aa2c79884998bce4baa6427d3ce7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/7df0293ab53049bbb865acbe7db749fe_SeqId_47_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp, totalSize=337.0 K 2023-02-09 09:54:32,351 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 26badea380234f70b06d65eb90830cfd, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840054 2023-02-09 09:54:32,352 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 92b6aa2c79884998bce4baa6427d3ce7, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240054 2023-02-09 09:54:32,352 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/a92f70dca41d459680717060df8c201a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/a92f70dca41d459680717060df8c201a 2023-02-09 09:54:32,352 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 7df0293ab53049bbb865acbe7db749fe_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840054 2023-02-09 09:54:32,353 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:32,354 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:32,354 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/5096e3e103244604bdffd85db9aca2e5 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/5096e3e103244604bdffd85db9aca2e5 2023-02-09 09:54:32,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/d397cb16a351444991784cee13a9ef60_SeqId_44_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/d397cb16a351444991784cee13a9ef60_SeqId_44_ 2023-02-09 09:54:32,356 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130986149e24c3e4207a2e2097856796ade to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130986149e24c3e4207a2e2097856796ade 2023-02-09 09:54:32,357 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120104aba48088714903bdf3d3a439a47920 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120104aba48088714903bdf3d3a439a47920 2023-02-09 09:54:32,358 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120547b39ea1d3b34504a11a3eb6d80dcfa7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120547b39ea1d3b34504a11a3eb6d80dcfa7 2023-02-09 09:54:32,359 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-02-09 09:54:32,359 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 09:54:32,359 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 09:54:32,359 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 09:54:32,359 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:32,360 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:54:32,362 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8700 sec 2023-02-09 09:54:32,377 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 651cd17c19c5ed7af78ba9deadfd9f44#family1#compaction#49 average throughput is 11.90 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:54:32,472 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=255 (was 251) - Thread LEAK? -, OpenFileDescriptor=605 (was 605), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=307 (was 359), ProcessCount=168 (was 168), AvailableMemoryMB=3401 (was 3528) 2023-02-09 09:54:32,485 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=255, OpenFileDescriptor=605, MaxFileDescriptor=60000, SystemLoadAverage=307, ProcessCount=168, AvailableMemoryMB=3400 2023-02-09 09:54:32,486 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:32,486 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:32,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 09:54:32,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:32,489 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:54:32,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-02-09 09:54:32,490 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:32,490 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:32,490 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:54:32,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 09:54:32,491 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:32,492 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea empty. 2023-02-09 09:54:32,492 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:32,492 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-02-09 09:54:32,591 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:32,591 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:32,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 09:54:32,792 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:32,792 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:32,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 09:54:32,900 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-09 09:54:32,901 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 71fdc83e3110212c7e72a4e2bc3c2dea, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:54:33,093 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:33,093 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:33,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 09:54:33,307 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:33,307 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1603): Closing 71fdc83e3110212c7e72a4e2bc3c2dea, disabling compactions & flushes 2023-02-09 09:54:33,307 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:33,307 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:33,307 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. after waiting 0 ms 2023-02-09 09:54:33,307 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:33,307 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:33,307 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1557): Region close journal for 71fdc83e3110212c7e72a4e2bc3c2dea: Waiting for close lock at 1675936473307Disabling compacts and flushes for region at 1675936473307Disabling writes for close at 1675936473307Writing region close event to WAL at 1675936473307Closed at 1675936473307 2023-02-09 09:54:33,310 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:54:33,310 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936473310"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936473310"}]},"ts":"1675936473310"} 2023-02-09 09:54:33,312 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 09:54:33,313 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:54:33,313 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936473313"}]},"ts":"1675936473313"} 2023-02-09 09:54:33,314 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-02-09 09:54:33,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, ASSIGN}] 2023-02-09 09:54:33,318 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, ASSIGN 2023-02-09 09:54:33,318 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:54:33,469 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=71fdc83e3110212c7e72a4e2bc3c2dea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:33,470 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936473469"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936473469"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936473469"}]},"ts":"1675936473469"} 2023-02-09 09:54:33,472 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 71fdc83e3110212c7e72a4e2bc3c2dea, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:33,594 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:33,594 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:33,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 09:54:33,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:33,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 71fdc83e3110212c7e72a4e2bc3c2dea, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:54:33,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:33,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:33,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:33,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:33,627 INFO [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:33,629 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:33,629 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:33,629 INFO [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 71fdc83e3110212c7e72a4e2bc3c2dea columnFamilyName family1 2023-02-09 09:54:33,631 INFO [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(310): Store=71fdc83e3110212c7e72a4e2bc3c2dea/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:33,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:33,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:33,633 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/.tmp/family1/27feb29b08924cf3a317da1141e52999 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/27feb29b08924cf3a317da1141e52999 2023-02-09 09:54:33,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:33,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:54:33,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 71fdc83e3110212c7e72a4e2bc3c2dea; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66531183, jitterRate=-0.008608117699623108}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:54:33,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 71fdc83e3110212c7e72a4e2bc3c2dea: Running coprocessor pre-open hook at 1675936473626Writing region info on filesystem at 1675936473626Initializing all the Stores at 1675936473627 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675936473627Cleaning up temporary data from old regions at 1675936473632 (+5 ms)Cleaning up detritus from prior splits at 1675936473633 (+1 ms)Running coprocessor post-open hooks at 1675936473637 (+4 ms)Region opened successfully at 1675936473637 2023-02-09 09:54:33,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea., pid=41, masterSystemTime=1675936473623 2023-02-09 09:54:33,638 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 651cd17c19c5ed7af78ba9deadfd9f44/family1 of 651cd17c19c5ed7af78ba9deadfd9f44 into 27feb29b08924cf3a317da1141e52999(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:54:33,638 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 651cd17c19c5ed7af78ba9deadfd9f44: 2023-02-09 09:54:33,638 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1675936472351; duration=1sec 2023-02-09 09:54:33,638 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:33,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:33,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:33,639 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=71fdc83e3110212c7e72a4e2bc3c2dea, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:33,640 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936473639"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936473639"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936473639"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936473639"}]},"ts":"1675936473639"} 2023-02-09 09:54:33,642 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-02-09 09:54:33,642 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 71fdc83e3110212c7e72a4e2bc3c2dea, server=jenkins-hbase4.apache.org,42719,1675936416061 in 170 msec 2023-02-09 09:54:33,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-02-09 09:54:33,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, ASSIGN in 326 msec 2023-02-09 09:54:33,644 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:54:33,645 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936473644"}]},"ts":"1675936473644"} 2023-02-09 09:54:33,645 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-02-09 09:54:33,648 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:54:33,649 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1610 sec 2023-02-09 09:54:33,729 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/26badea380234f70b06d65eb90830cfd, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b6aa2c79884998bce4baa6427d3ce7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/7df0293ab53049bbb865acbe7db749fe_SeqId_47_] to archive 2023-02-09 09:54:33,730 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:33,732 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/26badea380234f70b06d65eb90830cfd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/26badea380234f70b06d65eb90830cfd 2023-02-09 09:54:33,733 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b6aa2c79884998bce4baa6427d3ce7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/92b6aa2c79884998bce4baa6427d3ce7 2023-02-09 09:54:33,734 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/7df0293ab53049bbb865acbe7db749fe_SeqId_47_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/family1/7df0293ab53049bbb865acbe7db749fe_SeqId_47_ 2023-02-09 09:54:34,596 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:34,596 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:34,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 09:54:34,597 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-02-09 09:54:34,616 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:34,616 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:34,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:34,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:34,618 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:34,618 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:34,618 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 09:54:34,618 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 09:54:34,619 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:34,619 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:34,621 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:34,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:34,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:34,621 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:34,622 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:34,622 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:34,622 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:34,622 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:34,622 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:34,622 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 09:54:34,622 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:34,623 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 09:54:34,623 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:34,623 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:34,623 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:34,623 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. started... 2023-02-09 09:54:34,623 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 71fdc83e3110212c7e72a4e2bc3c2dea 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 09:54:35,040 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113043988e7bf85b4829b2a4a293fb45f9b1 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113043988e7bf85b4829b2a4a293fb45f9b1 2023-02-09 09:54:35,444 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/26c554175a5e4991b5ed2b3eabfc3c2c 2023-02-09 09:54:35,450 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/26c554175a5e4991b5ed2b3eabfc3c2c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/26c554175a5e4991b5ed2b3eabfc3c2c 2023-02-09 09:54:35,453 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/26c554175a5e4991b5ed2b3eabfc3c2c, entries=1000, sequenceid=5, filesize=191.4 K 2023-02-09 09:54:35,454 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 71fdc83e3110212c7e72a4e2bc3c2dea in 831ms, sequenceid=5, compaction requested=false 2023-02-09 09:54:35,454 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:35,455 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:35,455 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:35,455 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:35,455 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:35,455 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,455 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 09:54:35,455 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 09:54:35,457 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:35,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:35,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:35,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:35,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:35,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 09:54:35,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10a0c285[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:35,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 09:54:35,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,459 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,459 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 09:54:35,459 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 09:54:35,459 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:35,459 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,460 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:35,461 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,461 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 09:54:35,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:35,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:35,461 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:35,461 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 09:54:35,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:35,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:35,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:35,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:35,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 09:54:35,463 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 09:54:35,463 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:35,463 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:35,463 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,463 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:35,468 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,468 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,468 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:35,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:35,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:35,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:35,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:35,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:35,469 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,470 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,472 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,472 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:35,472 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:35,472 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:35,472 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 09:54:35,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:35,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:35,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:35,472 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,472 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:35,472 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,472 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,472 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,472 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:35,472 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:35,473 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:35,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:35,573 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:35,573 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:35,573 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:35,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:35,577 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:35,577 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:35,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:35,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:35,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,580 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:35,580 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:35,580 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 09:54:35,580 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 09:54:35,580 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,580 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,581 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,581 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:35,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:35,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:35,582 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:35,582 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:35,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:35,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,583 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 09:54:35,583 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:35,583 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 09:54:35,583 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:35,583 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 09:54:35,583 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:35,583 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:35,583 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:35,583 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. started... 2023-02-09 09:54:35,583 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 71fdc83e3110212c7e72a4e2bc3c2dea 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 09:54:35,597 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112806334c6c9ef2428b934097bfd334b768 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112806334c6c9ef2428b934097bfd334b768 2023-02-09 09:54:36,003 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/6f399046af12403fad0abe0d5c5dbfa3 2023-02-09 09:54:36,008 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/6f399046af12403fad0abe0d5c5dbfa3 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6f399046af12403fad0abe0d5c5dbfa3 2023-02-09 09:54:36,012 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6f399046af12403fad0abe0d5c5dbfa3, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 09:54:36,013 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 71fdc83e3110212c7e72a4e2bc3c2dea in 430ms, sequenceid=9, compaction requested=false 2023-02-09 09:54:36,013 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:36,013 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:36,013 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:36,013 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:36,013 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,013 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 09:54:36,013 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 09:54:36,014 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,014 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:36,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:36,015 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:36,015 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:36,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:36,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:36,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 09:54:36,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@109dadcd[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:36,016 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 09:54:36,016 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,018 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,018 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 09:54:36,018 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 09:54:36,018 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:36,018 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,018 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:36,020 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 09:54:36,020 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,020 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:36,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,020 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 09:54:36,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:36,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:36,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:36,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:36,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:36,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:36,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 09:54:36,022 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 09:54:36,022 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:36,022 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:36,022 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,022 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:36,023 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,023 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:36,023 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:36,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:36,024 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:36,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,025 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:36,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:36,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:36,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 09:54:36,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:36,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:36,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:36,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,032 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:36,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:36,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:36,033 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:36,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:36,033 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,033 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:36,033 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,033 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,133 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:36,133 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:36,133 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:36,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:36,137 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:36,137 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:36,138 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:36,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:36,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,139 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:36,139 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:36,139 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 09:54:36,139 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 09:54:36,140 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,140 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,141 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,141 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:36,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:36,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:36,141 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,142 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:36,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,142 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:36,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,142 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 09:54:36,142 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 09:54:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 09:54:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:36,143 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:36,143 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. started... 2023-02-09 09:54:36,143 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 71fdc83e3110212c7e72a4e2bc3c2dea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:36,555 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120101b996f1d09f4a45a2387ce3d4333e8b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120101b996f1d09f4a45a2387ce3d4333e8b 2023-02-09 09:54:36,961 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/0c6b7d2198c14edba322145979cd8e96 2023-02-09 09:54:36,965 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/0c6b7d2198c14edba322145979cd8e96 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/0c6b7d2198c14edba322145979cd8e96 2023-02-09 09:54:36,969 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/0c6b7d2198c14edba322145979cd8e96, entries=1, sequenceid=13, filesize=5.0 K 2023-02-09 09:54:36,970 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 71fdc83e3110212c7e72a4e2bc3c2dea in 827ms, sequenceid=13, compaction requested=true 2023-02-09 09:54:36,970 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:36,970 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:36,970 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:36,971 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:36,971 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,971 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 09:54:36,971 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 09:54:36,972 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,972 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:36,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:36,972 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,973 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:36,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:36,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:36,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:36,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 09:54:36,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b6a9362[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:36,974 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 09:54:36,974 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,975 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,975 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 09:54:36,975 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 09:54:36,975 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:36,975 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,975 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:36,976 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 09:54:36,976 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,976 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:36,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:36,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:36,976 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 09:54:36,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:36,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:36,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:36,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:36,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 09:54:36,979 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 09:54:36,979 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:36,979 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:36,979 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,979 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:36,982 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,982 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:36,982 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:36,982 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:36,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,983 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:36,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:36,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:36,985 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:36,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:36,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:36,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:36,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:36,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:36,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,985 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 09:54:36,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:36,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:36,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:36,986 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:36,986 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:37,086 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:37,086 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:37,086 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:37,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:37,090 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:37,090 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:37,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:37,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:37,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,093 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:37,093 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:37,093 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 09:54:37,093 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 09:54:37,093 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,093 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,094 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,094 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:37,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:37,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:37,094 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,094 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:37,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:37,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,095 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 09:54:37,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 09:54:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 09:54:37,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:37,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:37,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:37,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. started... 2023-02-09 09:54:37,096 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 71fdc83e3110212c7e72a4e2bc3c2dea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:37,508 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120581a07fc7ae104affa1836e73abffa36a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120581a07fc7ae104affa1836e73abffa36a 2023-02-09 09:54:37,649 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 09:54:37,913 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/6eba878eb6af4150a0e2a8b659c7e286 2023-02-09 09:54:37,919 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/6eba878eb6af4150a0e2a8b659c7e286 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6eba878eb6af4150a0e2a8b659c7e286 2023-02-09 09:54:37,922 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6eba878eb6af4150a0e2a8b659c7e286, entries=1, sequenceid=17, filesize=5.0 K 2023-02-09 09:54:37,923 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 71fdc83e3110212c7e72a4e2bc3c2dea in 827ms, sequenceid=17, compaction requested=true 2023-02-09 09:54:37,923 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:37,923 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:37,923 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:37,924 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:37,924 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,924 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 09:54:37,924 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 09:54:37,926 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,926 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:37,927 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,927 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:37,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 09:54:37,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58553059[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:37,928 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 09:54:37,928 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,929 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,929 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 09:54:37,929 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 09:54:37,929 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,929 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:37,929 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:37,930 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 09:54:37,930 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,931 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:37,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:37,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:37,931 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 09:54:37,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:37,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:37,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:37,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:37,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 09:54:37,933 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 09:54:37,933 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:37,933 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:37,933 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,933 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:37,934 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,934 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:37,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:37,934 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:37,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:37,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:37,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:37,935 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:37,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,936 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:37,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,938 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,938 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,938 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:37,938 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:37,938 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:37,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:37,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:37,938 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:37,938 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:37,939 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:37,939 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,939 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:37,939 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:37,939 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:37,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:37,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:38,039 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:38,039 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:38,039 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:38,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:38,043 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:38,044 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:38,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:38,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:38,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,046 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:38,046 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:38,046 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 09:54:38,046 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 09:54:38,046 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,046 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,047 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,047 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:38,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:38,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:38,048 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,048 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:38,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:38,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,048 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,049 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 09:54:38,049 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 09:54:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 09:54:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:38,049 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:38,049 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. started... 2023-02-09 09:54:38,049 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 71fdc83e3110212c7e72a4e2bc3c2dea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:38,461 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122833ac0fdbc4a84fa581ac6b39ed205057 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122833ac0fdbc4a84fa581ac6b39ed205057 2023-02-09 09:54:38,867 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/20d7a24893954a77854e9d0a87d3132c 2023-02-09 09:54:38,873 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/20d7a24893954a77854e9d0a87d3132c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/20d7a24893954a77854e9d0a87d3132c 2023-02-09 09:54:38,876 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/20d7a24893954a77854e9d0a87d3132c, entries=1, sequenceid=21, filesize=5.0 K 2023-02-09 09:54:38,877 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 71fdc83e3110212c7e72a4e2bc3c2dea in 828ms, sequenceid=21, compaction requested=true 2023-02-09 09:54:38,877 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:38,877 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:38,877 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:38,877 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:38,878 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,878 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 09:54:38,878 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 09:54:38,883 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,883 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:38,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:38,884 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,884 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:38,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:38,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:38,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:38,885 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 09:54:38,885 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e77653c[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:38,885 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 09:54:38,885 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,886 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,886 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 09:54:38,886 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 09:54:38,886 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:38,886 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,886 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:38,887 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,887 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 09:54:38,887 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:38,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:38,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:38,887 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 09:54:38,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:38,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:38,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:38,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:38,890 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 09:54:38,890 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:38,890 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:38,890 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 09:54:38,890 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,890 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:38,891 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,891 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:38,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:38,891 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:38,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:38,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:38,892 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:38,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:38,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,893 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:38,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,896 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,896 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,896 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,896 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:38,896 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,896 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:38,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 09:54:38,896 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:38,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:38,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:38,896 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:38,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:38,896 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:38,896 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:38,897 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:38,897 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:38,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:38,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:38,997 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:38,997 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:38,997 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:38,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:39,001 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:39,001 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:39,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:39,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:39,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,003 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:39,003 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:39,003 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 09:54:39,003 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 09:54:39,004 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,004 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,005 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:39,005 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:39,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:39,005 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,005 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:39,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,005 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:39,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,006 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 09:54:39,006 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,006 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 09:54:39,006 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:39,006 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 09:54:39,006 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:39,006 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:39,006 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:39,006 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. started... 2023-02-09 09:54:39,007 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 71fdc83e3110212c7e72a4e2bc3c2dea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:39,157 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:54:39,419 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123181f54ae1ed52491b8df64469432a540a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123181f54ae1ed52491b8df64469432a540a 2023-02-09 09:54:39,825 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/f0f61435a9d1410a97c3e0bafc36e877 2023-02-09 09:54:39,830 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/f0f61435a9d1410a97c3e0bafc36e877 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/f0f61435a9d1410a97c3e0bafc36e877 2023-02-09 09:54:39,833 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/f0f61435a9d1410a97c3e0bafc36e877, entries=1, sequenceid=25, filesize=5.0 K 2023-02-09 09:54:39,834 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 71fdc83e3110212c7e72a4e2bc3c2dea in 828ms, sequenceid=25, compaction requested=true 2023-02-09 09:54:39,834 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:39,834 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:39,834 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:39,834 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:39,834 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,834 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 09:54:39,834 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 09:54:39,836 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,836 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:39,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:39,836 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,836 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:39,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:39,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:39,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:39,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 09:54:39,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33f94afa[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:39,837 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 09:54:39,837 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,838 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,838 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 09:54:39,838 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 09:54:39,838 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,838 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:39,838 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:39,841 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 09:54:39,841 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:39,841 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,841 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 09:54:39,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:39,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:39,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:39,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:39,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:39,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:39,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 09:54:39,843 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 09:54:39,843 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:39,843 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:39,843 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,843 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:39,845 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,845 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:39,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,845 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:39,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:39,845 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:39,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,846 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:39,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:39,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:39,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:39,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:39,848 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:39,848 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:39,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:39,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:39,949 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:39,949 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:39,949 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:39,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:39,953 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:39,953 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:39,953 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:39,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:39,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,955 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:39,955 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:39,955 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 09:54:39,955 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 09:54:39,955 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,955 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,957 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,957 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:39,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:39,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:39,957 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:39,957 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:39,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:39,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,958 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,958 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 09:54:39,958 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:39,958 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 09:54:39,958 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:39,958 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 09:54:39,958 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:39,958 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:39,958 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:39,958 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. started... 2023-02-09 09:54:39,958 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 71fdc83e3110212c7e72a4e2bc3c2dea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:40,370 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e477a0231b3348c4a982ef6068a32142 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101e477a0231b3348c4a982ef6068a32142 2023-02-09 09:54:40,776 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/e7f722e38aa14c85b48c594d1d42a26d 2023-02-09 09:54:40,780 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/e7f722e38aa14c85b48c594d1d42a26d as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/e7f722e38aa14c85b48c594d1d42a26d 2023-02-09 09:54:40,785 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/e7f722e38aa14c85b48c594d1d42a26d, entries=1, sequenceid=29, filesize=5.0 K 2023-02-09 09:54:40,786 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 71fdc83e3110212c7e72a4e2bc3c2dea in 828ms, sequenceid=29, compaction requested=true 2023-02-09 09:54:40,786 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:40,786 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:40,786 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:40,786 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:40,786 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,786 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 09:54:40,786 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 09:54:40,789 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,789 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:40,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:40,789 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,789 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:40,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:40,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:40,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:40,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 09:54:40,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c872e64[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:40,790 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 09:54:40,790 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,792 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,792 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 09:54:40,792 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 09:54:40,792 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:40,792 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,792 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:40,793 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 09:54:40,793 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,793 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:40,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:40,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:40,793 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 09:54:40,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:40,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:40,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:40,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:40,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 09:54:40,795 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 09:54:40,795 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:40,795 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:40,796 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,796 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:40,800 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,800 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:40,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:40,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,801 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:40,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:40,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:40,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:40,801 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:40,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,802 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:40,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,804 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,804 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,804 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:40,804 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:40,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:40,804 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:40,804 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:40,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:40,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:40,804 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,804 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,804 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:40,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:40,804 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,805 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:40,805 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:40,905 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:40,905 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:40,905 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:40,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:40,909 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:40,909 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:40,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:40,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:40,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,911 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:40,911 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:40,911 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 09:54:40,911 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 09:54:40,912 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,912 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,913 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:40,913 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:40,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:40,913 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:40,913 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:40,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:40,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,914 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 09:54:40,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:40,914 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 09:54:40,914 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:40,914 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 09:54:40,914 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:40,914 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:40,914 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:40,915 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. started... 2023-02-09 09:54:40,915 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 71fdc83e3110212c7e72a4e2bc3c2dea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:41,327 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010368bfeb8512594b898c5abe8eb94b6a4a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010368bfeb8512594b898c5abe8eb94b6a4a 2023-02-09 09:54:41,733 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/c61b7b777c104c1e9f0c576efe4a5264 2023-02-09 09:54:41,738 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/c61b7b777c104c1e9f0c576efe4a5264 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/c61b7b777c104c1e9f0c576efe4a5264 2023-02-09 09:54:41,742 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/c61b7b777c104c1e9f0c576efe4a5264, entries=1, sequenceid=33, filesize=5.0 K 2023-02-09 09:54:41,742 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 71fdc83e3110212c7e72a4e2bc3c2dea in 827ms, sequenceid=33, compaction requested=true 2023-02-09 09:54:41,742 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:41,742 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:41,743 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:41,743 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:41,743 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,743 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 09:54:41,743 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 09:54:41,744 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,744 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:41,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:41,744 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,744 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:41,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:41,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:41,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:41,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 09:54:41,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4da10655[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:41,746 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 09:54:41,746 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,747 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,748 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 09:54:41,748 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 09:54:41,748 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:41,748 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,748 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:41,749 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,749 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 09:54:41,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:41,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:41,749 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:41,749 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 09:54:41,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:41,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:41,751 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 09:54:41,751 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 09:54:41,751 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:41,751 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:41,751 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,751 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:41,752 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,752 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:41,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:41,752 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:41,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:41,753 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:41,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:41,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:41,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,754 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:41,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:41,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:41,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:41,756 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:41,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,756 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:41,856 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:41,857 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:41,857 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:41,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:41,861 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:41,861 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:41,861 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:41,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:41,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,863 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:41,863 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:41,863 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 09:54:41,863 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 09:54:41,863 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,863 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,865 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:41,865 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:41,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:41,866 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:41,866 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:41,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:41,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,866 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,866 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 09:54:41,866 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:41,867 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 09:54:41,867 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:41,867 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 09:54:41,867 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:41,867 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:41,867 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:41,867 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. started... 2023-02-09 09:54:41,867 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 71fdc83e3110212c7e72a4e2bc3c2dea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:42,279 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509076c7b4274805744b381b2317939901d1d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201509076c7b4274805744b381b2317939901d1d 2023-02-09 09:54:42,685 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/630ffd3376bc458b938d32815033b3d2 2023-02-09 09:54:42,690 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/630ffd3376bc458b938d32815033b3d2 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/630ffd3376bc458b938d32815033b3d2 2023-02-09 09:54:42,693 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/630ffd3376bc458b938d32815033b3d2, entries=1, sequenceid=37, filesize=5.0 K 2023-02-09 09:54:42,694 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 71fdc83e3110212c7e72a4e2bc3c2dea in 827ms, sequenceid=37, compaction requested=true 2023-02-09 09:54:42,694 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:42,694 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:42,694 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:42,694 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:42,694 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,695 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 09:54:42,695 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 09:54:42,696 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,696 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:42,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:42,696 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,696 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:42,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:42,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:42,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:42,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 09:54:42,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@388ac458[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:42,697 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 09:54:42,698 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,698 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,699 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 09:54:42,699 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 09:54:42,699 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:42,699 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,699 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:42,701 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,701 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 09:54:42,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:42,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:42,701 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:42,701 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 09:54:42,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:42,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:42,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:42,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:42,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 09:54:42,703 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 09:54:42,703 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:42,703 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:42,703 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,703 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:42,705 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,705 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,705 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:42,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:42,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:42,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:42,705 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:42,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:42,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:42,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,706 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:42,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,711 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,711 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,711 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,711 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,711 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,711 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,711 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:42,711 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:42,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:42,711 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:42,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:42,711 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:42,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 09:54:42,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:42,711 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:42,712 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:42,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:42,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:42,812 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:42,812 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:42,812 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:42,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:42,816 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:42,816 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:42,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:42,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:42,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,818 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:42,818 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:42,818 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 09:54:42,818 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 09:54:42,819 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,819 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,820 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:42,820 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:42,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:42,820 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:42,820 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:42,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,821 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,821 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,821 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 09:54:42,821 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:42,821 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 09:54:42,821 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:42,821 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 09:54:42,822 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:42,822 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:42,822 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:42,822 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. started... 2023-02-09 09:54:42,822 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 71fdc83e3110212c7e72a4e2bc3c2dea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:43,234 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120678bde82edbe43e8bd86d9465b0111cb to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120678bde82edbe43e8bd86d9465b0111cb 2023-02-09 09:54:43,639 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/4baf814981014722a6e03b23d259cac0 2023-02-09 09:54:43,644 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/4baf814981014722a6e03b23d259cac0 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/4baf814981014722a6e03b23d259cac0 2023-02-09 09:54:43,648 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/4baf814981014722a6e03b23d259cac0, entries=1, sequenceid=41, filesize=5.0 K 2023-02-09 09:54:43,649 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 71fdc83e3110212c7e72a4e2bc3c2dea in 827ms, sequenceid=41, compaction requested=true 2023-02-09 09:54:43,649 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:43,649 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:43,649 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:43,649 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:43,649 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,649 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 09:54:43,649 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 09:54:43,650 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,650 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:43,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:43,651 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,651 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:43,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:43,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:43,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:43,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 09:54:43,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@415de729[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:43,652 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 09:54:43,652 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,654 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,654 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 09:54:43,654 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 09:54:43,654 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,654 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:43,654 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:43,655 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 09:54:43,655 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:43,655 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,655 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 09:54:43,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:43,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:43,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:43,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:43,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:43,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:43,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 09:54:43,658 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 09:54:43,658 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:43,658 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:43,658 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,658 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:43,659 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,659 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:43,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:43,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,659 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:43,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:43,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:43,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:43,659 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:43,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,661 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:43,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:43,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:43,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 09:54:43,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:43,663 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:43,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:43,663 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:43,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:43,763 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:43,764 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:43,764 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:43,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:43,766 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:43,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 09:54:43,768 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 09:54:43,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:43,769 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937083769 2023-02-09 09:54:43,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:54:43,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-09 09:54:43,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:43,771 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x52ffae0b to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:54:43,781 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@754c12e1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:54:43,781 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:44,619 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032a6e81298ddc45a1862b6e10d795b179 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601032a6e81298ddc45a1862b6e10d795b179 2023-02-09 09:54:44,620 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:44,632 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:54:44,633 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48678, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:54:44,639 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:44,639 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:44,639 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:54:44,641 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48140, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:54:44,650 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f50aea5c78db4562b8f4d6d036a67a0f first=Optional[r3] last=Optional[r6] 2023-02-09 09:54:44,650 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f50aea5c78db4562b8f4d6d036a67a0f}] 2023-02-09 09:54:44,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f50aea5c78db4562b8f4d6d036a67a0f for inclusion in 71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:44,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 09:54:44,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:54:44,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:44,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f50aea5c78db4562b8f4d6d036a67a0f to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__i4um9m2reedg0nhkigq11nf4kv35t9t4dhmeak4r1p6rs2c9g75ku0oi2jl45bs3/family1/f50aea5c78db4562b8f4d6d036a67a0f 2023-02-09 09:54:44,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__i4um9m2reedg0nhkigq11nf4kv35t9t4dhmeak4r1p6rs2c9g75ku0oi2jl45bs3/family1/f50aea5c78db4562b8f4d6d036a67a0f as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/d20987bea1de4b41ab6598981c4ec702_SeqId_44_ 2023-02-09 09:54:44,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__i4um9m2reedg0nhkigq11nf4kv35t9t4dhmeak4r1p6rs2c9g75ku0oi2jl45bs3/family1/f50aea5c78db4562b8f4d6d036a67a0f into 71fdc83e3110212c7e72a4e2bc3c2dea/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/d20987bea1de4b41ab6598981c4ec702_SeqId_44_ - updating store file list. 2023-02-09 09:54:44,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/d20987bea1de4b41ab6598981c4ec702_SeqId_44_ into 71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:44,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__i4um9m2reedg0nhkigq11nf4kv35t9t4dhmeak4r1p6rs2c9g75ku0oi2jl45bs3/family1/f50aea5c78db4562b8f4d6d036a67a0f into 71fdc83e3110212c7e72a4e2bc3c2dea/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/d20987bea1de4b41ab6598981c4ec702_SeqId_44_) 2023-02-09 09:54:44,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__i4um9m2reedg0nhkigq11nf4kv35t9t4dhmeak4r1p6rs2c9g75ku0oi2jl45bs3/family1/f50aea5c78db4562b8f4d6d036a67a0f 2023-02-09 09:54:44,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 09:54:44,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 09:54:44,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): 71fdc83e3110212c7e72a4e2bc3c2dea/family1 is initiating minor compaction 2023-02-09 09:54:44,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1a941701; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:44,668 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 71fdc83e3110212c7e72a4e2bc3c2dea/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:44,668 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6f399046af12403fad0abe0d5c5dbfa3, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/0c6b7d2198c14edba322145979cd8e96, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6eba878eb6af4150a0e2a8b659c7e286, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/20d7a24893954a77854e9d0a87d3132c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/f0f61435a9d1410a97c3e0bafc36e877, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/e7f722e38aa14c85b48c594d1d42a26d, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/c61b7b777c104c1e9f0c576efe4a5264, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/630ffd3376bc458b938d32815033b3d2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/4baf814981014722a6e03b23d259cac0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/d20987bea1de4b41ab6598981c4ec702_SeqId_44_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp, totalSize=50.4 K 2023-02-09 09:54:44,668 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 6f399046af12403fad0abe0d5c5dbfa3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 09:54:44,668 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 0c6b7d2198c14edba322145979cd8e96, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 09:54:44,668 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 6eba878eb6af4150a0e2a8b659c7e286, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 09:54:44,668 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 20d7a24893954a77854e9d0a87d3132c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 09:54:44,668 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting f0f61435a9d1410a97c3e0bafc36e877, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 09:54:44,668 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting e7f722e38aa14c85b48c594d1d42a26d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 09:54:44,668 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting c61b7b777c104c1e9f0c576efe4a5264, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 09:54:44,668 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 630ffd3376bc458b938d32815033b3d2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 09:54:44,668 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 4baf814981014722a6e03b23d259cac0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 09:54:44,668 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting d20987bea1de4b41ab6598981c4ec702_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 09:54:44,670 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:44,671 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:44,672 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122833ac0fdbc4a84fa581ac6b39ed205057 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122833ac0fdbc4a84fa581ac6b39ed205057 2023-02-09 09:54:44,674 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123181f54ae1ed52491b8df64469432a540a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123181f54ae1ed52491b8df64469432a540a 2023-02-09 09:54:44,675 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101e477a0231b3348c4a982ef6068a32142 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101e477a0231b3348c4a982ef6068a32142 2023-02-09 09:54:44,676 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010368bfeb8512594b898c5abe8eb94b6a4a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010368bfeb8512594b898c5abe8eb94b6a4a 2023-02-09 09:54:44,676 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-02-09 09:54:44,676 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 09:54:44,676 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:44,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:44,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:44,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:44,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:44,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:44,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:44,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:44,700 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 71fdc83e3110212c7e72a4e2bc3c2dea#family1#compaction#60 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:54:45,134 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120543c2da3f961a4d5aa9c17806a5851618 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120543c2da3f961a4d5aa9c17806a5851618 2023-02-09 09:54:45,135 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:45,151 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:45,151 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:45,158 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9e4f790643ea40b6985e0e56f5148f5f first=Optional[r00] last=Optional[r2] 2023-02-09 09:54:45,159 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9e4f790643ea40b6985e0e56f5148f5f}] 2023-02-09 09:54:45,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9e4f790643ea40b6985e0e56f5148f5f for inclusion in 71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:45,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 09:54:45,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:54:45,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:45,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9e4f790643ea40b6985e0e56f5148f5f to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__id62d1u12ojo1vur8o1gjuhvt67bqcporvn49udqcfi3d6eqf08p7l86modhmgh4/family1/9e4f790643ea40b6985e0e56f5148f5f 2023-02-09 09:54:45,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__id62d1u12ojo1vur8o1gjuhvt67bqcporvn49udqcfi3d6eqf08p7l86modhmgh4/family1/9e4f790643ea40b6985e0e56f5148f5f as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/586163ebe69549b69caf96e7a81f40c1_SeqId_46_ 2023-02-09 09:54:45,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__id62d1u12ojo1vur8o1gjuhvt67bqcporvn49udqcfi3d6eqf08p7l86modhmgh4/family1/9e4f790643ea40b6985e0e56f5148f5f into 71fdc83e3110212c7e72a4e2bc3c2dea/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/586163ebe69549b69caf96e7a81f40c1_SeqId_46_ - updating store file list. 2023-02-09 09:54:45,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/586163ebe69549b69caf96e7a81f40c1_SeqId_46_ into 71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:45,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__id62d1u12ojo1vur8o1gjuhvt67bqcporvn49udqcfi3d6eqf08p7l86modhmgh4/family1/9e4f790643ea40b6985e0e56f5148f5f into 71fdc83e3110212c7e72a4e2bc3c2dea/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/586163ebe69549b69caf96e7a81f40c1_SeqId_46_) 2023-02-09 09:54:45,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__id62d1u12ojo1vur8o1gjuhvt67bqcporvn49udqcfi3d6eqf08p7l86modhmgh4/family1/9e4f790643ea40b6985e0e56f5148f5f 2023-02-09 09:54:45,173 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:45,174 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:45,175 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113043988e7bf85b4829b2a4a293fb45f9b1 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113043988e7bf85b4829b2a4a293fb45f9b1 2023-02-09 09:54:45,177 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120101b996f1d09f4a45a2387ce3d4333e8b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120101b996f1d09f4a45a2387ce3d4333e8b 2023-02-09 09:54:45,178 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120581a07fc7ae104affa1836e73abffa36a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120581a07fc7ae104affa1836e73abffa36a 2023-02-09 09:54:45,178 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-02-09 09:54:45,178 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 09:54:45,178 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:45,178 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:45,178 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:45,179 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 09:54:45,181 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.4130 sec 2023-02-09 09:54:45,259 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:45,259 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:45,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 09:54:45,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:45,526 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/fcd2ebb1d4b44199a6e78832bb55e55a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/fcd2ebb1d4b44199a6e78832bb55e55a 2023-02-09 09:54:45,531 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 71fdc83e3110212c7e72a4e2bc3c2dea/family1 of 71fdc83e3110212c7e72a4e2bc3c2dea into fcd2ebb1d4b44199a6e78832bb55e55a(size=6.8 K), total size for store is 390.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:54:45,531 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:45,531 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1675936484668; duration=0sec 2023-02-09 09:54:45,531 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:45,635 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6f399046af12403fad0abe0d5c5dbfa3, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/0c6b7d2198c14edba322145979cd8e96, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6eba878eb6af4150a0e2a8b659c7e286, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/20d7a24893954a77854e9d0a87d3132c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/f0f61435a9d1410a97c3e0bafc36e877, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/e7f722e38aa14c85b48c594d1d42a26d, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/c61b7b777c104c1e9f0c576efe4a5264, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/630ffd3376bc458b938d32815033b3d2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/4baf814981014722a6e03b23d259cac0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/d20987bea1de4b41ab6598981c4ec702_SeqId_44_] to archive 2023-02-09 09:54:45,636 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:45,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6f399046af12403fad0abe0d5c5dbfa3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6f399046af12403fad0abe0d5c5dbfa3 2023-02-09 09:54:45,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/0c6b7d2198c14edba322145979cd8e96 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/0c6b7d2198c14edba322145979cd8e96 2023-02-09 09:54:45,640 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6eba878eb6af4150a0e2a8b659c7e286 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/6eba878eb6af4150a0e2a8b659c7e286 2023-02-09 09:54:45,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/20d7a24893954a77854e9d0a87d3132c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/20d7a24893954a77854e9d0a87d3132c 2023-02-09 09:54:45,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/f0f61435a9d1410a97c3e0bafc36e877 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/f0f61435a9d1410a97c3e0bafc36e877 2023-02-09 09:54:45,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/e7f722e38aa14c85b48c594d1d42a26d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/e7f722e38aa14c85b48c594d1d42a26d 2023-02-09 09:54:45,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/c61b7b777c104c1e9f0c576efe4a5264 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/c61b7b777c104c1e9f0c576efe4a5264 2023-02-09 09:54:45,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/630ffd3376bc458b938d32815033b3d2 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/630ffd3376bc458b938d32815033b3d2 2023-02-09 09:54:45,646 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/4baf814981014722a6e03b23d259cac0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/4baf814981014722a6e03b23d259cac0 2023-02-09 09:54:45,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/d20987bea1de4b41ab6598981c4ec702_SeqId_44_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/d20987bea1de4b41ab6598981c4ec702_SeqId_44_ 2023-02-09 09:54:45,670 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-09 09:54:45,671 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-09 09:54:45,671 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-09 09:54:45,672 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-09 09:54:45,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, REOPEN/MOVE}] 2023-02-09 09:54:45,675 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, REOPEN/MOVE 2023-02-09 09:54:45,676 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=71fdc83e3110212c7e72a4e2bc3c2dea, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:45,676 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936485676"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936485676"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936485676"}]},"ts":"1675936485676"} 2023-02-09 09:54:45,677 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 71fdc83e3110212c7e72a4e2bc3c2dea, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:45,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:45,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 71fdc83e3110212c7e72a4e2bc3c2dea, disabling compactions & flushes 2023-02-09 09:54:45,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:45,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:45,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. after waiting 0 ms 2023-02-09 09:54:45,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:45,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-02-09 09:54:45,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:45,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 71fdc83e3110212c7e72a4e2bc3c2dea: Waiting for close lock at 1675936485830Running coprocessor pre-close hooks at 1675936485830Disabling compacts and flushes for region at 1675936485830Disabling writes for close at 1675936485830Writing region close event to WAL at 1675936485831 (+1 ms)Running coprocessor post-close hooks at 1675936485835 (+4 ms)Closed at 1675936485836 (+1 ms) 2023-02-09 09:54:45,836 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 71fdc83e3110212c7e72a4e2bc3c2dea to self. 2023-02-09 09:54:45,838 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:45,838 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=71fdc83e3110212c7e72a4e2bc3c2dea, regionState=CLOSED 2023-02-09 09:54:45,838 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936485838"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936485838"}]},"ts":"1675936485838"} 2023-02-09 09:54:45,840 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-09 09:54:45,841 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 71fdc83e3110212c7e72a4e2bc3c2dea, server=jenkins-hbase4.apache.org,42719,1675936416061 in 162 msec 2023-02-09 09:54:45,841 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=true 2023-02-09 09:54:45,991 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=71fdc83e3110212c7e72a4e2bc3c2dea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:45,992 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936485991"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936485991"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936485991"}]},"ts":"1675936485991"} 2023-02-09 09:54:45,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 71fdc83e3110212c7e72a4e2bc3c2dea, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:46,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:46,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 71fdc83e3110212c7e72a4e2bc3c2dea, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:54:46,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:46,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:46,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:46,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:46,149 INFO [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:46,151 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:46,151 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:46,151 INFO [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 71fdc83e3110212c7e72a4e2bc3c2dea columnFamilyName family1 2023-02-09 09:54:46,156 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/26c554175a5e4991b5ed2b3eabfc3c2c 2023-02-09 09:54:46,160 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/586163ebe69549b69caf96e7a81f40c1_SeqId_46_ 2023-02-09 09:54:46,163 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/fcd2ebb1d4b44199a6e78832bb55e55a 2023-02-09 09:54:46,163 INFO [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(310): Store=71fdc83e3110212c7e72a4e2bc3c2dea/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:46,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:46,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:46,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:46,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 71fdc83e3110212c7e72a4e2bc3c2dea; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67274109, jitterRate=0.002462342381477356}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:54:46,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 71fdc83e3110212c7e72a4e2bc3c2dea: Running coprocessor pre-open hook at 1675936486148Writing region info on filesystem at 1675936486148Initializing all the Stores at 1675936486149 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675936486149Cleaning up temporary data from old regions at 1675936486166 (+17 ms)Cleaning up detritus from prior splits at 1675936486166Running coprocessor post-open hooks at 1675936486168 (+2 ms)Region opened successfully at 1675936486168 2023-02-09 09:54:46,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea., pid=47, masterSystemTime=1675936486145 2023-02-09 09:54:46,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:46,170 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 09:54:46,171 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 09:54:46,172 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1912): 71fdc83e3110212c7e72a4e2bc3c2dea/family1 is initiating minor compaction (all files) 2023-02-09 09:54:46,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:46,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:46,172 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 71fdc83e3110212c7e72a4e2bc3c2dea/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:46,172 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/26c554175a5e4991b5ed2b3eabfc3c2c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/fcd2ebb1d4b44199a6e78832bb55e55a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/586163ebe69549b69caf96e7a81f40c1_SeqId_46_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp, totalSize=390.1 K 2023-02-09 09:54:46,172 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=71fdc83e3110212c7e72a4e2bc3c2dea, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:46,172 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936486172"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936486172"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936486172"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936486172"}]},"ts":"1675936486172"} 2023-02-09 09:54:46,172 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 26c554175a5e4991b5ed2b3eabfc3c2c, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840054 2023-02-09 09:54:46,173 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting fcd2ebb1d4b44199a6e78832bb55e55a, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240054 2023-02-09 09:54:46,173 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 586163ebe69549b69caf96e7a81f40c1_SeqId_46_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878840054 2023-02-09 09:54:46,175 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-02-09 09:54:46,175 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 71fdc83e3110212c7e72a4e2bc3c2dea, server=jenkins-hbase4.apache.org,42719,1675936416061 in 180 msec 2023-02-09 09:54:46,177 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-09 09:54:46,177 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, REOPEN/MOVE in 500 msec 2023-02-09 09:54:46,179 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-02-09 09:54:46,179 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 505 msec 2023-02-09 09:54:46,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 920 msec 2023-02-09 09:54:46,181 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:46,182 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:46,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-02-09 09:54:46,182 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:46,182 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:46,183 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:46,183 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:46,184 INFO [Listener at localhost/42557] compactions.TestMobCompactor(278): alter status finished 2023-02-09 09:54:46,185 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:46,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 09:54:46,187 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 09:54:46,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:46,187 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937086187 2023-02-09 09:54:46,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:54:46,189 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 71fdc83e3110212c7e72a4e2bc3c2dea#family1#compaction#61 average throughput is 22.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:54:46,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-09 09:54:46,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:46,189 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x59e1407d to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:54:46,196 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4caf8aba, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:54:46,196 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:46,197 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:46,197 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:46,197 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:46,197 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:47,021 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128292bafb2f38e419f94f39665ddea3c4f to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128292bafb2f38e419f94f39665ddea3c4f 2023-02-09 09:54:47,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:47,034 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:54:47,035 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48694, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:54:47,041 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:47,041 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:47,041 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:54:47,043 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48142, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:54:47,051 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/864325b6b33f4f64b47a885a231bec6a first=Optional[r05] last=Optional[r8] 2023-02-09 09:54:47,052 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=50 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/864325b6b33f4f64b47a885a231bec6a}] 2023-02-09 09:54:47,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/864325b6b33f4f64b47a885a231bec6a for inclusion in 71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:47,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-09 09:54:47,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:54:47,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:47,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/864325b6b33f4f64b47a885a231bec6a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__295ln9bj1r83nf25kek022aem9arpfg9290tn38ednfq32q60os89ff5e45mula7/family1/864325b6b33f4f64b47a885a231bec6a 2023-02-09 09:54:47,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__295ln9bj1r83nf25kek022aem9arpfg9290tn38ednfq32q60os89ff5e45mula7/family1/864325b6b33f4f64b47a885a231bec6a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/cce5a676268a49d18101d91fd49dfaba_SeqId_52_ 2023-02-09 09:54:47,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__295ln9bj1r83nf25kek022aem9arpfg9290tn38ednfq32q60os89ff5e45mula7/family1/864325b6b33f4f64b47a885a231bec6a into 71fdc83e3110212c7e72a4e2bc3c2dea/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/cce5a676268a49d18101d91fd49dfaba_SeqId_52_ - updating store file list. 2023-02-09 09:54:47,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/cce5a676268a49d18101d91fd49dfaba_SeqId_52_ into 71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:47,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__295ln9bj1r83nf25kek022aem9arpfg9290tn38ednfq32q60os89ff5e45mula7/family1/864325b6b33f4f64b47a885a231bec6a into 71fdc83e3110212c7e72a4e2bc3c2dea/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/cce5a676268a49d18101d91fd49dfaba_SeqId_52_) 2023-02-09 09:54:47,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__295ln9bj1r83nf25kek022aem9arpfg9290tn38ednfq32q60os89ff5e45mula7/family1/864325b6b33f4f64b47a885a231bec6a 2023-02-09 09:54:47,069 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:47,070 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:47,071 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120678bde82edbe43e8bd86d9465b0111cb to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120678bde82edbe43e8bd86d9465b0111cb 2023-02-09 09:54:47,073 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112806334c6c9ef2428b934097bfd334b768 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112806334c6c9ef2428b934097bfd334b768 2023-02-09 09:54:47,073 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-02-09 09:54:47,073 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 09:54:47,073 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:47,073 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:47,073 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:47,073 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:47,073 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:47,075 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 09:54:47,076 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 890 msec 2023-02-09 09:54:47,127 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:47,127 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:47,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 09:54:47,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:47,411 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/.tmp/family1/75816c7739464306a722fbe4e5e62fb9 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/75816c7739464306a722fbe4e5e62fb9 2023-02-09 09:54:47,416 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 71fdc83e3110212c7e72a4e2bc3c2dea/family1 of 71fdc83e3110212c7e72a4e2bc3c2dea into 75816c7739464306a722fbe4e5e62fb9(size=193.0 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:54:47,416 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 71fdc83e3110212c7e72a4e2bc3c2dea: 2023-02-09 09:54:47,416 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea., storeName=71fdc83e3110212c7e72a4e2bc3c2dea/family1, priority=13, startTime=1675936486169; duration=1sec 2023-02-09 09:54:47,416 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:54:47,430 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/26c554175a5e4991b5ed2b3eabfc3c2c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/fcd2ebb1d4b44199a6e78832bb55e55a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/586163ebe69549b69caf96e7a81f40c1_SeqId_46_] to archive 2023-02-09 09:54:47,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:54:47,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/26c554175a5e4991b5ed2b3eabfc3c2c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/26c554175a5e4991b5ed2b3eabfc3c2c 2023-02-09 09:54:47,434 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/fcd2ebb1d4b44199a6e78832bb55e55a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/fcd2ebb1d4b44199a6e78832bb55e55a 2023-02-09 09:54:47,435 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/586163ebe69549b69caf96e7a81f40c1_SeqId_46_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/586163ebe69549b69caf96e7a81f40c1_SeqId_46_ 2023-02-09 09:54:47,538 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-09 09:54:47,539 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-09 09:54:47,539 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-09 09:54:47,541 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-09 09:54:47,544 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, REOPEN/MOVE}] 2023-02-09 09:54:47,545 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, REOPEN/MOVE 2023-02-09 09:54:47,546 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=71fdc83e3110212c7e72a4e2bc3c2dea, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:47,546 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936487546"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936487546"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936487546"}]},"ts":"1675936487546"} 2023-02-09 09:54:47,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 71fdc83e3110212c7e72a4e2bc3c2dea, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:47,699 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:47,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 71fdc83e3110212c7e72a4e2bc3c2dea, disabling compactions & flushes 2023-02-09 09:54:47,699 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:47,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:47,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. after waiting 0 ms 2023-02-09 09:54:47,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:47,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-02-09 09:54:47,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:47,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 71fdc83e3110212c7e72a4e2bc3c2dea: Waiting for close lock at 1675936487699Running coprocessor pre-close hooks at 1675936487699Disabling compacts and flushes for region at 1675936487699Disabling writes for close at 1675936487699Writing region close event to WAL at 1675936487702 (+3 ms)Running coprocessor post-close hooks at 1675936487705 (+3 ms)Closed at 1675936487706 (+1 ms) 2023-02-09 09:54:47,706 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 71fdc83e3110212c7e72a4e2bc3c2dea to self. 2023-02-09 09:54:47,707 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:47,708 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=71fdc83e3110212c7e72a4e2bc3c2dea, regionState=CLOSED 2023-02-09 09:54:47,708 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936487708"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936487708"}]},"ts":"1675936487708"} 2023-02-09 09:54:47,710 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-02-09 09:54:47,710 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 71fdc83e3110212c7e72a4e2bc3c2dea, server=jenkins-hbase4.apache.org,42719,1675936416061 in 162 msec 2023-02-09 09:54:47,711 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=true 2023-02-09 09:54:47,861 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=71fdc83e3110212c7e72a4e2bc3c2dea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:47,861 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936487861"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936487861"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936487861"}]},"ts":"1675936487861"} 2023-02-09 09:54:47,863 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 71fdc83e3110212c7e72a4e2bc3c2dea, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:48,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:48,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 71fdc83e3110212c7e72a4e2bc3c2dea, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:54:48,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:48,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:48,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:48,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:48,019 INFO [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:48,020 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:48,020 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1 2023-02-09 09:54:48,020 INFO [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 71fdc83e3110212c7e72a4e2bc3c2dea columnFamilyName family1 2023-02-09 09:54:48,025 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/75816c7739464306a722fbe4e5e62fb9 2023-02-09 09:54:48,027 DEBUG [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/family1/cce5a676268a49d18101d91fd49dfaba_SeqId_52_ 2023-02-09 09:54:48,028 INFO [StoreOpener-71fdc83e3110212c7e72a4e2bc3c2dea-1] regionserver.HStore(310): Store=71fdc83e3110212c7e72a4e2bc3c2dea/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:48,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:48,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:48,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:54:48,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 71fdc83e3110212c7e72a4e2bc3c2dea; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61492186, jitterRate=-0.08369502425193787}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:54:48,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 71fdc83e3110212c7e72a4e2bc3c2dea: Running coprocessor pre-open hook at 1675936488018Writing region info on filesystem at 1675936488018Initializing all the Stores at 1675936488018Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675936488019 (+1 ms)Cleaning up temporary data from old regions at 1675936488030 (+11 ms)Cleaning up detritus from prior splits at 1675936488030Running coprocessor post-open hooks at 1675936488032 (+2 ms)Region opened successfully at 1675936488032 2023-02-09 09:54:48,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea., pid=53, masterSystemTime=1675936488014 2023-02-09 09:54:48,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:48,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:54:48,035 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=71fdc83e3110212c7e72a4e2bc3c2dea, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:48,035 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675936488035"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936488035"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936488035"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936488035"}]},"ts":"1675936488035"} 2023-02-09 09:54:48,038 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-02-09 09:54:48,038 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 71fdc83e3110212c7e72a4e2bc3c2dea, server=jenkins-hbase4.apache.org,42719,1675936416061 in 173 msec 2023-02-09 09:54:48,039 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-09 09:54:48,039 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=71fdc83e3110212c7e72a4e2bc3c2dea, REOPEN/MOVE in 494 msec 2023-02-09 09:54:48,041 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-09 09:54:48,041 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 498 msec 2023-02-09 09:54:48,042 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 912 msec 2023-02-09 09:54:48,042 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:48,043 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:48,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 09:54:48,043 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:48,043 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:48,044 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:48,044 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:48,044 INFO [Listener at localhost/42557] compactions.TestMobCompactor(278): alter status finished 2023-02-09 09:54:48,045 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 09:54:48,046 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 09:54:48,047 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 09:54:48,047 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:48,047 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937088047 2023-02-09 09:54:48,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:54:48,049 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-09 09:54:48,049 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:48,049 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x606b73bf to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:54:48,053 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5021df52, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:54:48,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:48,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:48,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:48,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:48,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:48,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:48,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:48,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 09:54:48,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 09:54:48,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:48,054 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 09:54:48,055 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 8 msec 2023-02-09 09:54:48,109 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=246 (was 255), OpenFileDescriptor=618 (was 605) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=247 (was 307), ProcessCount=168 (was 168), AvailableMemoryMB=3402 (was 3400) - AvailableMemoryMB LEAK? - 2023-02-09 09:54:48,120 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=246, OpenFileDescriptor=618, MaxFileDescriptor=60000, SystemLoadAverage=247, ProcessCount=168, AvailableMemoryMB=3401 2023-02-09 09:54:48,120 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:48,121 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:48,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 09:54:48,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:48,124 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:54:48,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-02-09 09:54:48,125 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:48,125 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:48,125 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:54:48,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 09:54:48,126 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:54:48,127 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f empty. 2023-02-09 09:54:48,127 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:54:48,127 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-02-09 09:54:48,226 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:48,226 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:48,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 09:54:48,427 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:48,427 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:48,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 09:54:48,535 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-09 09:54:48,536 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7671): creating {ENCODED => bf230a91f2dd4768974bd854b33e048f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:54:48,728 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:48,729 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:48,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 09:54:48,942 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:48,942 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1603): Closing bf230a91f2dd4768974bd854b33e048f, disabling compactions & flushes 2023-02-09 09:54:48,942 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:48,942 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:48,942 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. after waiting 0 ms 2023-02-09 09:54:48,942 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:48,942 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:48,942 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1557): Region close journal for bf230a91f2dd4768974bd854b33e048f: Waiting for close lock at 1675936488942Disabling compacts and flushes for region at 1675936488942Disabling writes for close at 1675936488942Writing region close event to WAL at 1675936488942Closed at 1675936488942 2023-02-09 09:54:48,944 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:54:48,945 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936488945"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936488945"}]},"ts":"1675936488945"} 2023-02-09 09:54:48,946 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 09:54:48,947 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:54:48,947 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936488947"}]},"ts":"1675936488947"} 2023-02-09 09:54:48,948 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-02-09 09:54:48,951 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, ASSIGN}] 2023-02-09 09:54:48,953 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, ASSIGN 2023-02-09 09:54:48,953 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:54:49,104 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=bf230a91f2dd4768974bd854b33e048f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:49,104 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936489104"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936489104"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936489104"}]},"ts":"1675936489104"} 2023-02-09 09:54:49,106 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure bf230a91f2dd4768974bd854b33e048f, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:54:49,230 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:49,230 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:49,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 09:54:49,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:49,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bf230a91f2dd4768974bd854b33e048f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:54:49,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:54:49,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:54:49,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:54:49,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:54:49,262 INFO [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:54:49,263 DEBUG [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:54:49,263 DEBUG [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:54:49,264 INFO [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf230a91f2dd4768974bd854b33e048f columnFamilyName family1 2023-02-09 09:54:49,265 INFO [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] regionserver.HStore(310): Store=bf230a91f2dd4768974bd854b33e048f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:54:49,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:54:49,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:54:49,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:54:49,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:54:49,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bf230a91f2dd4768974bd854b33e048f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66087409, jitterRate=-0.015220865607261658}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:54:49,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bf230a91f2dd4768974bd854b33e048f: Running coprocessor pre-open hook at 1675936489261Writing region info on filesystem at 1675936489261Initializing all the Stores at 1675936489261Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675936489262 (+1 ms)Cleaning up temporary data from old regions at 1675936489267 (+5 ms)Cleaning up detritus from prior splits at 1675936489267Running coprocessor post-open hooks at 1675936489271 (+4 ms)Region opened successfully at 1675936489271 2023-02-09 09:54:49,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f., pid=57, masterSystemTime=1675936489257 2023-02-09 09:54:49,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:49,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:49,274 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=bf230a91f2dd4768974bd854b33e048f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:49,274 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936489274"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936489274"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936489274"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936489274"}]},"ts":"1675936489274"} 2023-02-09 09:54:49,276 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-09 09:54:49,276 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure bf230a91f2dd4768974bd854b33e048f, server=jenkins-hbase4.apache.org,42719,1675936416061 in 169 msec 2023-02-09 09:54:49,278 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-02-09 09:54:49,278 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, ASSIGN in 325 msec 2023-02-09 09:54:49,278 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:54:49,278 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936489278"}]},"ts":"1675936489278"} 2023-02-09 09:54:49,279 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-02-09 09:54:49,282 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:54:49,283 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1600 sec 2023-02-09 09:54:49,603 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:50,231 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:50,231 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:50,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 09:54:50,232 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-02-09 09:54:50,249 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:50,249 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:50,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:50,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:50,251 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:50,251 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:50,251 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 09:54:50,251 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 09:54:50,252 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:50,252 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:50,253 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:50,253 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:50,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:50,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:50,253 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:50,253 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:50,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:50,254 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:50,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:50,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:50,254 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:50,254 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:50,254 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 09:54:50,254 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:50,254 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 09:54:50,256 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:50,256 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:50,256 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:50,256 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. started... 2023-02-09 09:54:50,257 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bf230a91f2dd4768974bd854b33e048f 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 09:54:50,678 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511300b460629ca3940498e5ea007f2877e2d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511300b460629ca3940498e5ea007f2877e2d 2023-02-09 09:54:51,082 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/2ec73fb0cf4144bc97b6f26f44dd6a9c 2023-02-09 09:54:51,086 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/2ec73fb0cf4144bc97b6f26f44dd6a9c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/2ec73fb0cf4144bc97b6f26f44dd6a9c 2023-02-09 09:54:51,091 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/2ec73fb0cf4144bc97b6f26f44dd6a9c, entries=1000, sequenceid=5, filesize=190.5 K 2023-02-09 09:54:51,092 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for bf230a91f2dd4768974bd854b33e048f in 835ms, sequenceid=5, compaction requested=false 2023-02-09 09:54:51,092 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:54:51,092 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:51,092 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:51,092 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:51,092 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,092 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 09:54:51,092 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 09:54:51,094 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,094 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:51,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:51,094 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,094 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:51,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:51,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 09:54:51,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44ac0568[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:51,096 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 09:54:51,096 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,096 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 09:54:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 09:54:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:51,097 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,097 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:51,099 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,099 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 09:54:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:51,099 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:51,099 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 09:54:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:51,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 09:54:51,101 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 09:54:51,101 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:51,101 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:51,101 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,101 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:51,102 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,102 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:51,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,103 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:51,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:51,103 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,104 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,109 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,109 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:51,109 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,109 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:51,109 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,109 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:51,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:51,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:51,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:51,109 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:51,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,110 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:51,110 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:51,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:51,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:51,210 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:51,210 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:51,210 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:51,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:51,214 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:51,214 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:51,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:51,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:51,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,217 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:51,217 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:51,217 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 09:54:51,217 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 09:54:51,217 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,217 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,218 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,218 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:51,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:51,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:51,219 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:51,219 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:51,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:51,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,219 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,219 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 09:54:51,219 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:51,220 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 09:54:51,220 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:51,220 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 09:54:51,220 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:51,220 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:51,220 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:51,220 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. started... 2023-02-09 09:54:51,220 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bf230a91f2dd4768974bd854b33e048f 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 09:54:51,381 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:54:51,631 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112805121abea9304b7b9ddd776f29b1ade2 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112805121abea9304b7b9ddd776f29b1ade2 2023-02-09 09:54:52,036 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/4c0106183c82415aa5be6551ff7c3eb4 2023-02-09 09:54:52,041 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/4c0106183c82415aa5be6551ff7c3eb4 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/4c0106183c82415aa5be6551ff7c3eb4 2023-02-09 09:54:52,045 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/4c0106183c82415aa5be6551ff7c3eb4, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 09:54:52,046 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for bf230a91f2dd4768974bd854b33e048f in 826ms, sequenceid=9, compaction requested=false 2023-02-09 09:54:52,046 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:54:52,046 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:52,046 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:52,046 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:52,046 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,046 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 09:54:52,046 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 09:54:52,048 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,048 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:52,048 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,048 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:52,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 09:54:52,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fa2006[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:52,049 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 09:54:52,049 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,052 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,052 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 09:54:52,052 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 09:54:52,052 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:52,052 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,052 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:52,053 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,054 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 09:54:52,054 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:52,054 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 09:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:52,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:52,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 09:54:52,056 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 09:54:52,056 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:52,056 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:52,056 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,056 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:52,057 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,058 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:52,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,058 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:52,058 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:52,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,059 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:52,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,061 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,061 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:52,061 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:52,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:52,061 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,061 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:52,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:52,061 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:52,061 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,062 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,062 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,062 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:52,062 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,062 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:52,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:52,162 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:52,162 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:52,162 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:52,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:52,166 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:52,166 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:52,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:52,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:52,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,169 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:52,169 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:52,169 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 09:54:52,169 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 09:54:52,169 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,169 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,172 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,172 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:52,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:52,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:52,172 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:52,172 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:52,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:52,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,173 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,173 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 09:54:52,173 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:52,173 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 09:54:52,173 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:52,173 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 09:54:52,173 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:52,173 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:52,173 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:52,173 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. started... 2023-02-09 09:54:52,173 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bf230a91f2dd4768974bd854b33e048f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:52,587 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120131c5189fdc7743bd8f0c016f24607a62 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120131c5189fdc7743bd8f0c016f24607a62 2023-02-09 09:54:52,992 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/129c150229de4dee95621e83f0569117 2023-02-09 09:54:52,997 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/129c150229de4dee95621e83f0569117 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/129c150229de4dee95621e83f0569117 2023-02-09 09:54:53,001 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/129c150229de4dee95621e83f0569117, entries=1, sequenceid=13, filesize=5.0 K 2023-02-09 09:54:53,002 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf230a91f2dd4768974bd854b33e048f in 829ms, sequenceid=13, compaction requested=true 2023-02-09 09:54:53,002 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:54:53,002 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:53,002 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:53,002 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:53,002 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,002 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 09:54:53,002 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 09:54:53,003 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,003 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:53,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:53,004 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,004 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:53,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:53,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:53,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:53,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 09:54:53,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6fbc8c65[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:53,005 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 09:54:53,005 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,006 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,006 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 09:54:53,006 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 09:54:53,006 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:53,006 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,007 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:53,007 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,007 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 09:54:53,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:53,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:53,007 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:53,008 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 09:54:53,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:53,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:53,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:53,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:53,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 09:54:53,010 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 09:54:53,010 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:53,010 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:53,010 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,010 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:53,012 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,012 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:53,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:53,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,012 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:53,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:53,012 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,013 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:53,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,015 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,015 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,015 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:53,015 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:53,015 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 09:54:53,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:53,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:53,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:53,015 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:53,015 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,015 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:53,016 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:53,016 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,016 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,016 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:53,016 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:53,116 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:53,116 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:53,116 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:53,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:53,120 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:53,120 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:53,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:53,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:53,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,122 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:53,122 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:53,122 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 09:54:53,123 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 09:54:53,123 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,123 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,124 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,124 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:53,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:53,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:53,124 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,124 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:53,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:53,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,125 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,125 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 09:54:53,125 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,125 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 09:54:53,125 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:53,126 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 09:54:53,126 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:53,126 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:53,126 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:53,126 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. started... 2023-02-09 09:54:53,126 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bf230a91f2dd4768974bd854b33e048f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:53,537 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120571d63f8a9a3743ba9886e1a808937c7a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120571d63f8a9a3743ba9886e1a808937c7a 2023-02-09 09:54:53,943 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/8604ece64a6b46aaafb14d80b9c79426 2023-02-09 09:54:53,948 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/8604ece64a6b46aaafb14d80b9c79426 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/8604ece64a6b46aaafb14d80b9c79426 2023-02-09 09:54:53,952 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/8604ece64a6b46aaafb14d80b9c79426, entries=1, sequenceid=17, filesize=5.0 K 2023-02-09 09:54:53,953 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf230a91f2dd4768974bd854b33e048f in 827ms, sequenceid=17, compaction requested=true 2023-02-09 09:54:53,953 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:54:53,953 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:53,953 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:53,953 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:53,953 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,953 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 09:54:53,953 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 09:54:53,955 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:53,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:53,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:53,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:53,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:53,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 09:54:53,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41f65a14[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:53,956 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 09:54:53,956 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,962 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,962 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 09:54:53,962 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 09:54:53,962 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:53,962 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,962 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:53,963 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 09:54:53,963 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,963 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:53,963 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 09:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 09:54:53,966 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 09:54:53,966 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:53,966 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:53,966 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,966 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:53,967 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,967 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:53,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,967 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:53,968 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,969 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:53,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:53,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,971 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:53,971 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:53,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:53,971 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:54,071 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:54,072 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:54,072 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:54,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:54,076 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:54,076 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:54,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:54,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:54,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,078 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:54,078 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:54,078 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 09:54:54,078 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 09:54:54,078 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,078 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,080 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,080 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:54,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:54,081 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,081 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,082 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 09:54:54,082 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,082 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 09:54:54,082 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:54,082 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 09:54:54,082 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:54,082 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:54,082 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:54,082 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. started... 2023-02-09 09:54:54,082 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bf230a91f2dd4768974bd854b33e048f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:54,494 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c0300327c3d9432dba3cea5758d5bbfb to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228c0300327c3d9432dba3cea5758d5bbfb 2023-02-09 09:54:54,899 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/35055a2c227f4190a98c41e81c9f29d2 2023-02-09 09:54:54,904 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/35055a2c227f4190a98c41e81c9f29d2 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/35055a2c227f4190a98c41e81c9f29d2 2023-02-09 09:54:54,907 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/35055a2c227f4190a98c41e81c9f29d2, entries=1, sequenceid=21, filesize=5.0 K 2023-02-09 09:54:54,908 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf230a91f2dd4768974bd854b33e048f in 826ms, sequenceid=21, compaction requested=true 2023-02-09 09:54:54,908 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:54:54,908 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:54,908 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:54,908 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:54,908 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,908 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 09:54:54,908 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 09:54:54,910 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:54,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 09:54:54,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65d58007[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:54,912 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 09:54:54,912 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,913 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,913 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 09:54:54,913 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 09:54:54,913 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,913 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:54,913 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:54,914 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,914 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 09:54:54,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,914 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:54,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:54,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:54,914 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 09:54:54,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:54,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:54,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:54,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:54,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 09:54:54,916 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 09:54:54,917 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:54,917 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:54,917 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,917 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:54,919 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,919 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:54,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:54,919 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:54,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:54,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:54,919 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:54,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:54,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,920 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:54,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,922 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,922 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:54,922 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,922 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:54,922 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 09:54:54,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:54,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:54,922 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:54,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:54,922 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:54,923 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:54,923 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,923 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,923 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:54,923 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:54,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:54,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:54,923 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:55,023 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:55,024 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:55,024 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:55,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:55,028 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:55,028 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:55,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:55,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:55,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,030 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:55,031 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:55,031 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 09:54:55,031 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 09:54:55,031 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,031 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,032 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:55,032 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:55,032 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,032 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:55,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:55,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 09:54:55,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,033 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 09:54:55,033 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:55,033 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 09:54:55,034 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:55,034 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:55,034 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:55,034 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. started... 2023-02-09 09:54:55,034 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bf230a91f2dd4768974bd854b33e048f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:55,445 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c0b72b1945a04967952e13332073f545 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c0b72b1945a04967952e13332073f545 2023-02-09 09:54:55,850 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/a3f1ec52a2724036a2a2ac1ccfe765d9 2023-02-09 09:54:55,854 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/a3f1ec52a2724036a2a2ac1ccfe765d9 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/a3f1ec52a2724036a2a2ac1ccfe765d9 2023-02-09 09:54:55,858 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/a3f1ec52a2724036a2a2ac1ccfe765d9, entries=1, sequenceid=25, filesize=5.0 K 2023-02-09 09:54:55,859 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf230a91f2dd4768974bd854b33e048f in 825ms, sequenceid=25, compaction requested=true 2023-02-09 09:54:55,859 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:54:55,859 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 09:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 09:54:55,861 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:55,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:55,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:55,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:55,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:55,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 09:54:55,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35ff66ab[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:55,862 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 09:54:55,863 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,864 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,865 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 09:54:55,865 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 09:54:55,865 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:55,865 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,865 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:55,866 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,866 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 09:54:55,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:55,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:55,866 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:55,866 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 09:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 09:54:55,868 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 09:54:55,869 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:55,869 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:55,869 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,869 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:55,870 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,870 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:55,870 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,870 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,871 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,877 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,877 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:55,877 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:55,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:55,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:55,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:55,877 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:55,877 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,877 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:55,878 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,878 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:55,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 09:54:55,878 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:55,878 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:55,878 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:55,878 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,978 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:55,978 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:55,978 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:55,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:55,982 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:55,983 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:55,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:55,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:55,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,985 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:55,985 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:55,985 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 09:54:55,985 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 09:54:55,985 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,985 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,989 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,989 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:55,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:55,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:55,989 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:55,989 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:55,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:55,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,990 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 09:54:55,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:55,990 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 09:54:55,991 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:55,991 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 09:54:55,991 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:55,991 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:55,991 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:55,991 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. started... 2023-02-09 09:54:55,991 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bf230a91f2dd4768974bd854b33e048f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:56,402 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010100722d929cd84dbe9b491bbc2208109e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010100722d929cd84dbe9b491bbc2208109e 2023-02-09 09:54:56,807 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/c4c9205de98c499da97bc1a07b6be43e 2023-02-09 09:54:56,812 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/c4c9205de98c499da97bc1a07b6be43e as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/c4c9205de98c499da97bc1a07b6be43e 2023-02-09 09:54:56,815 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/c4c9205de98c499da97bc1a07b6be43e, entries=1, sequenceid=29, filesize=5.0 K 2023-02-09 09:54:56,816 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf230a91f2dd4768974bd854b33e048f in 825ms, sequenceid=29, compaction requested=true 2023-02-09 09:54:56,816 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:54:56,816 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:56,816 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:56,816 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:56,816 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,816 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 09:54:56,816 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 09:54:56,818 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,818 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:56,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:56,818 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,818 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:56,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:56,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:56,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:56,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 09:54:56,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e2b0bd5[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:56,820 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 09:54:56,820 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,821 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,821 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 09:54:56,821 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 09:54:56,821 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:56,821 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,821 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:56,827 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 09:54:56,827 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:56,827 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,827 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 09:54:56,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:56,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:56,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:56,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:56,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:56,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 09:54:56,829 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 09:54:56,830 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:56,830 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:56,830 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,830 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:56,831 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,831 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:56,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:56,831 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:56,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:56,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:56,831 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:56,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:56,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,832 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:56,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,834 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:56,834 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:56,834 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,834 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:56,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:56,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:56,834 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 09:54:56,834 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:56,835 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,835 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:56,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:56,835 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:56,835 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,835 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,835 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:56,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:56,935 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:56,935 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:56,935 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:56,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:56,940 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:56,940 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:56,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:56,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:56,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,942 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:56,942 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:56,942 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 09:54:56,942 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 09:54:56,943 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,943 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,944 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,944 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:56,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:56,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:56,944 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:56,944 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:56,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:56,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,946 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 09:54:56,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:56,946 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 09:54:56,946 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:56,946 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 09:54:56,951 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:56,951 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:56,952 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:56,952 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. started... 2023-02-09 09:54:56,952 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bf230a91f2dd4768974bd854b33e048f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:57,367 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037ed2d0f4823446babac6d1fad0ac5109 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601037ed2d0f4823446babac6d1fad0ac5109 2023-02-09 09:54:57,773 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/aba95f9d01e4433eb3a09349e5afd4be 2023-02-09 09:54:57,777 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/aba95f9d01e4433eb3a09349e5afd4be as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/aba95f9d01e4433eb3a09349e5afd4be 2023-02-09 09:54:57,781 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/aba95f9d01e4433eb3a09349e5afd4be, entries=1, sequenceid=33, filesize=5.0 K 2023-02-09 09:54:57,782 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf230a91f2dd4768974bd854b33e048f in 830ms, sequenceid=33, compaction requested=true 2023-02-09 09:54:57,782 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:54:57,782 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 09:54:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 09:54:57,784 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,784 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:57,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:57,784 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,784 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:57,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:57,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:57,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 09:54:57,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d7ec0fe[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:57,785 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 09:54:57,785 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,786 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,787 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 09:54:57,787 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,787 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:57,787 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 09:54:57,787 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:57,788 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,788 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 09:54:57,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:57,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:57,788 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:57,788 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 09:54:57,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:57,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:57,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:57,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:57,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 09:54:57,790 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 09:54:57,790 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:57,790 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:57,790 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,790 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:57,793 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,793 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:57,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:57,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,793 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:57,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:57,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:57,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:57,793 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:57,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,795 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:57,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:57,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:57,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:57,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:57,797 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,797 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:57,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:57,897 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:57,898 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:57,898 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:57,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:57,902 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:57,902 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:57,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:57,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:57,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,905 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:57,905 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:57,905 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 09:54:57,905 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 09:54:57,905 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,905 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,906 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,906 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:57,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:57,907 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:57,907 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,908 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,908 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 09:54:57,908 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:57,908 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 09:54:57,908 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:57,908 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 09:54:57,909 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:57,909 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:57,909 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:57,909 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. started... 2023-02-09 09:54:57,910 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bf230a91f2dd4768974bd854b33e048f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:58,329 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090769ea222e8b0d413b86c70d80d509e397 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015090769ea222e8b0d413b86c70d80d509e397 2023-02-09 09:54:58,734 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/9169236e4ae040d791000af1b85fabdb 2023-02-09 09:54:58,756 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:54:58,756 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/9169236e4ae040d791000af1b85fabdb as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/9169236e4ae040d791000af1b85fabdb 2023-02-09 09:54:58,756 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x52ffae0b to 127.0.0.1:64170 2023-02-09 09:54:58,756 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:54:58,757 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:54:58,757 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x59e1407d to 127.0.0.1:64170 2023-02-09 09:54:58,757 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:54:58,757 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x606b73bf to 127.0.0.1:64170 2023-02-09 09:54:58,757 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:54:58,757 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:54:58,757 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1f9e2030 to 127.0.0.1:64170 2023-02-09 09:54:58,757 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:54:58,761 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/9169236e4ae040d791000af1b85fabdb, entries=1, sequenceid=37, filesize=5.0 K 2023-02-09 09:54:58,762 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf230a91f2dd4768974bd854b33e048f in 852ms, sequenceid=37, compaction requested=true 2023-02-09 09:54:58,762 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:54:58,762 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:58,762 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:58,762 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:58,762 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,762 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 09:54:58,762 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 09:54:58,763 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,763 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:58,764 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,764 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:58,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 09:54:58,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@462b4ee[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:58,765 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 09:54:58,765 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,766 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,766 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 09:54:58,766 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 09:54:58,766 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:58,766 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,766 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:58,769 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 09:54:58,769 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,769 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:58,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:58,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:58,769 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 09:54:58,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:58,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:58,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:58,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:58,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 09:54:58,771 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 09:54:58,771 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:58,771 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:58,771 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,771 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:58,772 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,772 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:58,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:58,772 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:58,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:58,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:58,773 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:58,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,774 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:58,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:58,776 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:58,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:58,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:58,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:58,776 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:58,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:58,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:58,877 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:58,877 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:58,877 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:58,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:58,881 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:58,881 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:58,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:54:58,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 09:54:58,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,883 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:58,883 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:58,883 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 09:54:58,883 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 09:54:58,884 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,884 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,886 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:58,886 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:58,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:58,886 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:58,886 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:54:58,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:54:58,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 09:54:58,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:58,887 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 09:54:58,887 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:54:58,887 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 09:54:58,887 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:54:58,887 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:54:58,887 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:58,887 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. started... 2023-02-09 09:54:58,888 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bf230a91f2dd4768974bd854b33e048f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:54:59,299 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e9d8fcfb8e074ae485104570a819ffa9 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120e9d8fcfb8e074ae485104570a819ffa9 2023-02-09 09:54:59,704 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/341a780d2b254c02823a39cf5ec91b54 2023-02-09 09:54:59,709 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/341a780d2b254c02823a39cf5ec91b54 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/341a780d2b254c02823a39cf5ec91b54 2023-02-09 09:54:59,714 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/341a780d2b254c02823a39cf5ec91b54, entries=1, sequenceid=41, filesize=5.0 K 2023-02-09 09:54:59,714 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bf230a91f2dd4768974bd854b33e048f in 826ms, sequenceid=41, compaction requested=true 2023-02-09 09:54:59,715 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:54:59,715 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:54:59,715 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:54:59,715 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:54:59,715 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,715 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 09:54:59,715 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 09:54:59,717 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,717 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:59,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:59,717 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,717 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:54:59,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:59,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:59,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:59,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 09:54:59,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ae640f7[Count = 0] remaining members to acquire global barrier 2023-02-09 09:54:59,718 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 09:54:59,718 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,719 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,720 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 09:54:59,720 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 09:54:59,720 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,720 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:54:59,720 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:54:59,721 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 09:54:59,721 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,721 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:59,721 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 09:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:59,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:59,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:59,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:54:59,723 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 09:54:59,723 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 09:54:59,723 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:54:59,723 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:54:59,723 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,723 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:54:59,724 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,724 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:54:59,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,725 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:54:59,725 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:54:59,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,726 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:54:59,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,729 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,729 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:54:59,729 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 09:54:59,729 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,729 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:54:59,729 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:54:59,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:54:59,729 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:54:59,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:54:59,729 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 09:54:59,729 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:54:59,729 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,730 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,730 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 09:54:59,730 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:54:59,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:54:59,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:54:59,830 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:54:59,830 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:54:59,830 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:54:59,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:54:59,832 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:54:59,833 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 09:54:59,834 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 09:54:59,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:54:59,835 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937099835 2023-02-09 09:54:59,837 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:54:59,838 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-09 09:54:59,838 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:54:59,838 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x1db827c7 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:54:59,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c7d18fd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:54:59,842 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:00,684 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034a89fa5502234a469d2988889984e065 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601034a89fa5502234a469d2988889984e065 2023-02-09 09:55:00,685 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:00,697 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:55:00,699 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43226, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:55:00,704 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:00,704 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:00,704 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:55:00,705 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36130, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:55:00,713 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/98246d30a2a949ec90e0e606d020d469 first=Optional[r3] last=Optional[r6] 2023-02-09 09:55:00,713 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/98246d30a2a949ec90e0e606d020d469}] 2023-02-09 09:55:00,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/98246d30a2a949ec90e0e606d020d469 for inclusion in bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:55:00,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 09:55:00,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:55:00,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:55:00,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/98246d30a2a949ec90e0e606d020d469 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__m2m59v2adjjjloo2or268qve1mapg0j4qjtglhu693h7cb7ou31fjgunp1fgjuh5/family1/98246d30a2a949ec90e0e606d020d469 2023-02-09 09:55:00,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__m2m59v2adjjjloo2or268qve1mapg0j4qjtglhu693h7cb7ou31fjgunp1fgjuh5/family1/98246d30a2a949ec90e0e606d020d469 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/d3fef6113f7f46848e9049703f8f4d89_SeqId_44_ 2023-02-09 09:55:00,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__m2m59v2adjjjloo2or268qve1mapg0j4qjtglhu693h7cb7ou31fjgunp1fgjuh5/family1/98246d30a2a949ec90e0e606d020d469 into bf230a91f2dd4768974bd854b33e048f/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/d3fef6113f7f46848e9049703f8f4d89_SeqId_44_ - updating store file list. 2023-02-09 09:55:00,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/d3fef6113f7f46848e9049703f8f4d89_SeqId_44_ into bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:55:00,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__m2m59v2adjjjloo2or268qve1mapg0j4qjtglhu693h7cb7ou31fjgunp1fgjuh5/family1/98246d30a2a949ec90e0e606d020d469 into bf230a91f2dd4768974bd854b33e048f/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/d3fef6113f7f46848e9049703f8f4d89_SeqId_44_) 2023-02-09 09:55:00,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__m2m59v2adjjjloo2or268qve1mapg0j4qjtglhu693h7cb7ou31fjgunp1fgjuh5/family1/98246d30a2a949ec90e0e606d020d469 2023-02-09 09:55:00,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 09:55:00,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 09:55:00,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): bf230a91f2dd4768974bd854b33e048f/family1 is initiating minor compaction 2023-02-09 09:55:00,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3824b80; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:55:00,728 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bf230a91f2dd4768974bd854b33e048f/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:00,728 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/4c0106183c82415aa5be6551ff7c3eb4, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/129c150229de4dee95621e83f0569117, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/8604ece64a6b46aaafb14d80b9c79426, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/35055a2c227f4190a98c41e81c9f29d2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/a3f1ec52a2724036a2a2ac1ccfe765d9, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/c4c9205de98c499da97bc1a07b6be43e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/aba95f9d01e4433eb3a09349e5afd4be, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/9169236e4ae040d791000af1b85fabdb, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/341a780d2b254c02823a39cf5ec91b54, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/d3fef6113f7f46848e9049703f8f4d89_SeqId_44_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp, totalSize=50.3 K 2023-02-09 09:55:00,728 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 4c0106183c82415aa5be6551ff7c3eb4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 09:55:00,728 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 129c150229de4dee95621e83f0569117, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 09:55:00,728 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 8604ece64a6b46aaafb14d80b9c79426, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 09:55:00,728 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 35055a2c227f4190a98c41e81c9f29d2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 09:55:00,729 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting a3f1ec52a2724036a2a2ac1ccfe765d9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 09:55:00,729 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting c4c9205de98c499da97bc1a07b6be43e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 09:55:00,729 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting aba95f9d01e4433eb3a09349e5afd4be, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 09:55:00,729 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 9169236e4ae040d791000af1b85fabdb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 09:55:00,729 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 341a780d2b254c02823a39cf5ec91b54, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 09:55:00,729 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting d3fef6113f7f46848e9049703f8f4d89_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 09:55:00,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:00,730 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:00,732 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228c0300327c3d9432dba3cea5758d5bbfb to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228c0300327c3d9432dba3cea5758d5bbfb 2023-02-09 09:55:00,733 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c0b72b1945a04967952e13332073f545 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c0b72b1945a04967952e13332073f545 2023-02-09 09:55:00,734 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010100722d929cd84dbe9b491bbc2208109e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010100722d929cd84dbe9b491bbc2208109e 2023-02-09 09:55:00,735 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601037ed2d0f4823446babac6d1fad0ac5109 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601037ed2d0f4823446babac6d1fad0ac5109 2023-02-09 09:55:00,735 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-02-09 09:55:00,735 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 09:55:00,735 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:00,735 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:00,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:00,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:00,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:00,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:00,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:00,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:00,776 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): bf230a91f2dd4768974bd854b33e048f#family1#compaction#72 average throughput is 0.32 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:01,603 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/60c4b1dceee64ed282beaa93989082f8 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/60c4b1dceee64ed282beaa93989082f8 2023-02-09 09:55:01,608 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in bf230a91f2dd4768974bd854b33e048f/family1 of bf230a91f2dd4768974bd854b33e048f into 60c4b1dceee64ed282beaa93989082f8(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:55:01,608 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:55:01,608 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1675936500728; duration=0sec 2023-02-09 09:55:01,608 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:55:01,608 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054e644b530c80442ca04c32b5232ca9d5 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512054e644b530c80442ca04c32b5232ca9d5 2023-02-09 09:55:01,609 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:01,624 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:01,625 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:01,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/4c0106183c82415aa5be6551ff7c3eb4, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/129c150229de4dee95621e83f0569117, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/8604ece64a6b46aaafb14d80b9c79426, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/35055a2c227f4190a98c41e81c9f29d2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/a3f1ec52a2724036a2a2ac1ccfe765d9, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/c4c9205de98c499da97bc1a07b6be43e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/aba95f9d01e4433eb3a09349e5afd4be, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/9169236e4ae040d791000af1b85fabdb, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/341a780d2b254c02823a39cf5ec91b54, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/d3fef6113f7f46848e9049703f8f4d89_SeqId_44_] to archive 2023-02-09 09:55:01,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:01,641 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c05b7cc1a3e340559b028bd47aeec6f9 first=Optional[r00] last=Optional[r2] 2023-02-09 09:55:01,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/4c0106183c82415aa5be6551ff7c3eb4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/4c0106183c82415aa5be6551ff7c3eb4 2023-02-09 09:55:01,641 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c05b7cc1a3e340559b028bd47aeec6f9}] 2023-02-09 09:55:01,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/129c150229de4dee95621e83f0569117 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/129c150229de4dee95621e83f0569117 2023-02-09 09:55:01,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c05b7cc1a3e340559b028bd47aeec6f9 for inclusion in bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:55:01,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/8604ece64a6b46aaafb14d80b9c79426 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/8604ece64a6b46aaafb14d80b9c79426 2023-02-09 09:55:01,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/35055a2c227f4190a98c41e81c9f29d2 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/35055a2c227f4190a98c41e81c9f29d2 2023-02-09 09:55:01,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 09:55:01,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:55:01,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:55:01,646 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/a3f1ec52a2724036a2a2ac1ccfe765d9 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/a3f1ec52a2724036a2a2ac1ccfe765d9 2023-02-09 09:55:01,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c05b7cc1a3e340559b028bd47aeec6f9 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8fqa8p3cuvv1alvo927o1r4k6d0v6ekeriqe7k3cl69qsdp1nt15sd0q564b6jit/family1/c05b7cc1a3e340559b028bd47aeec6f9 2023-02-09 09:55:01,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/c4c9205de98c499da97bc1a07b6be43e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/c4c9205de98c499da97bc1a07b6be43e 2023-02-09 09:55:01,648 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/aba95f9d01e4433eb3a09349e5afd4be to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/aba95f9d01e4433eb3a09349e5afd4be 2023-02-09 09:55:01,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8fqa8p3cuvv1alvo927o1r4k6d0v6ekeriqe7k3cl69qsdp1nt15sd0q564b6jit/family1/c05b7cc1a3e340559b028bd47aeec6f9 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/80cf21d53bd748d5812f0e0a6f44e782_SeqId_47_ 2023-02-09 09:55:01,650 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/9169236e4ae040d791000af1b85fabdb to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/9169236e4ae040d791000af1b85fabdb 2023-02-09 09:55:01,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8fqa8p3cuvv1alvo927o1r4k6d0v6ekeriqe7k3cl69qsdp1nt15sd0q564b6jit/family1/c05b7cc1a3e340559b028bd47aeec6f9 into bf230a91f2dd4768974bd854b33e048f/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/80cf21d53bd748d5812f0e0a6f44e782_SeqId_47_ - updating store file list. 2023-02-09 09:55:01,651 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/341a780d2b254c02823a39cf5ec91b54 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/341a780d2b254c02823a39cf5ec91b54 2023-02-09 09:55:01,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/d3fef6113f7f46848e9049703f8f4d89_SeqId_44_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/d3fef6113f7f46848e9049703f8f4d89_SeqId_44_ 2023-02-09 09:55:01,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/80cf21d53bd748d5812f0e0a6f44e782_SeqId_47_ into bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:55:01,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8fqa8p3cuvv1alvo927o1r4k6d0v6ekeriqe7k3cl69qsdp1nt15sd0q564b6jit/family1/c05b7cc1a3e340559b028bd47aeec6f9 into bf230a91f2dd4768974bd854b33e048f/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/80cf21d53bd748d5812f0e0a6f44e782_SeqId_47_) 2023-02-09 09:55:01,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8fqa8p3cuvv1alvo927o1r4k6d0v6ekeriqe7k3cl69qsdp1nt15sd0q564b6jit/family1/c05b7cc1a3e340559b028bd47aeec6f9 2023-02-09 09:55:01,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 09:55:01,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 09:55:01,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): bf230a91f2dd4768974bd854b33e048f/family1 is initiating minor compaction (all files) 2023-02-09 09:55:01,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3d30198c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:55:01,655 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bf230a91f2dd4768974bd854b33e048f/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:01,655 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/2ec73fb0cf4144bc97b6f26f44dd6a9c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/60c4b1dceee64ed282beaa93989082f8, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/80cf21d53bd748d5812f0e0a6f44e782_SeqId_47_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp, totalSize=388.1 K 2023-02-09 09:55:01,656 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 2ec73fb0cf4144bc97b6f26f44dd6a9c, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840054 2023-02-09 09:55:01,656 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 60c4b1dceee64ed282beaa93989082f8, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240054 2023-02-09 09:55:01,656 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 80cf21d53bd748d5812f0e0a6f44e782_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840054 2023-02-09 09:55:01,657 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:01,658 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:01,659 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511300b460629ca3940498e5ea007f2877e2d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511300b460629ca3940498e5ea007f2877e2d 2023-02-09 09:55:01,660 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120131c5189fdc7743bd8f0c016f24607a62 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120131c5189fdc7743bd8f0c016f24607a62 2023-02-09 09:55:01,662 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120571d63f8a9a3743ba9886e1a808937c7a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120571d63f8a9a3743ba9886e1a808937c7a 2023-02-09 09:55:01,662 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-02-09 09:55:01,662 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 09:55:01,662 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:01,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:01,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:55:01,663 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 09:55:01,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8300 sec 2023-02-09 09:55:01,674 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): bf230a91f2dd4768974bd854b33e048f#family1#compaction#73 average throughput is 22.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:01,725 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:01,725 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:01,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 09:55:01,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:55:02,136 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-09 09:55:02,137 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-09 09:55:02,137 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-09 09:55:02,139 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-09 09:55:02,141 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, REOPEN/MOVE}] 2023-02-09 09:55:02,142 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, REOPEN/MOVE 2023-02-09 09:55:02,143 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=bf230a91f2dd4768974bd854b33e048f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:02,143 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936502142"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936502142"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936502142"}]},"ts":"1675936502142"} 2023-02-09 09:55:02,144 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure bf230a91f2dd4768974bd854b33e048f, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:02,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:02,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bf230a91f2dd4768974bd854b33e048f, disabling compactions & flushes 2023-02-09 09:55:02,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:02,894 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/.tmp/family1/8897bb3c6ea14347874dd27118059142 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/8897bb3c6ea14347874dd27118059142 2023-02-09 09:55:02,898 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in bf230a91f2dd4768974bd854b33e048f/family1 of bf230a91f2dd4768974bd854b33e048f into 8897bb3c6ea14347874dd27118059142(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:02,898 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:55:02,898 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:02,898 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1675936501655; duration=1sec 2023-02-09 09:55:02,898 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:02,898 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. after waiting 0 ms 2023-02-09 09:55:02,898 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:55:02,898 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:02,901 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/2ec73fb0cf4144bc97b6f26f44dd6a9c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/60c4b1dceee64ed282beaa93989082f8, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/80cf21d53bd748d5812f0e0a6f44e782_SeqId_47_] to archive 2023-02-09 09:55:02,902 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:02,903 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/2ec73fb0cf4144bc97b6f26f44dd6a9c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/2ec73fb0cf4144bc97b6f26f44dd6a9c 2023-02-09 09:55:02,904 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/60c4b1dceee64ed282beaa93989082f8 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/60c4b1dceee64ed282beaa93989082f8 2023-02-09 09:55:02,905 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/80cf21d53bd748d5812f0e0a6f44e782_SeqId_47_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/80cf21d53bd748d5812f0e0a6f44e782_SeqId_47_ 2023-02-09 09:55:02,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 09:55:02,910 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:02,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bf230a91f2dd4768974bd854b33e048f: Waiting for close lock at 1675936502296Running coprocessor pre-close hooks at 1675936502296Disabling compacts and flushes for region at 1675936502296Disabling writes for close at 1675936502898 (+602 ms)Writing region close event to WAL at 1675936502907 (+9 ms)Running coprocessor post-close hooks at 1675936502909 (+2 ms)Closed at 1675936502910 (+1 ms) 2023-02-09 09:55:02,910 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: bf230a91f2dd4768974bd854b33e048f to self. 2023-02-09 09:55:02,911 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:02,912 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=bf230a91f2dd4768974bd854b33e048f, regionState=CLOSED 2023-02-09 09:55:02,912 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936502912"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936502912"}]},"ts":"1675936502912"} 2023-02-09 09:55:02,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-02-09 09:55:02,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure bf230a91f2dd4768974bd854b33e048f, server=jenkins-hbase4.apache.org,42719,1675936416061 in 769 msec 2023-02-09 09:55:02,914 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=true 2023-02-09 09:55:03,065 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=bf230a91f2dd4768974bd854b33e048f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:03,065 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936503065"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936503065"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936503065"}]},"ts":"1675936503065"} 2023-02-09 09:55:03,066 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure bf230a91f2dd4768974bd854b33e048f, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:03,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:03,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bf230a91f2dd4768974bd854b33e048f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:55:03,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:03,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:03,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:03,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:03,223 INFO [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:03,223 DEBUG [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:55:03,223 DEBUG [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:55:03,224 INFO [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf230a91f2dd4768974bd854b33e048f columnFamilyName family1 2023-02-09 09:55:03,228 DEBUG [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/8897bb3c6ea14347874dd27118059142 2023-02-09 09:55:03,228 INFO [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] regionserver.HStore(310): Store=bf230a91f2dd4768974bd854b33e048f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:03,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:03,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:03,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:03,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bf230a91f2dd4768974bd854b33e048f; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70898321, jitterRate=0.05646730959415436}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:55:03,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bf230a91f2dd4768974bd854b33e048f: Running coprocessor pre-open hook at 1675936503222Writing region info on filesystem at 1675936503222Initializing all the Stores at 1675936503222Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675936503222Cleaning up temporary data from old regions at 1675936503230 (+8 ms)Cleaning up detritus from prior splits at 1675936503231 (+1 ms)Running coprocessor post-open hooks at 1675936503233 (+2 ms)Region opened successfully at 1675936503233 2023-02-09 09:55:03,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f., pid=63, masterSystemTime=1675936503218 2023-02-09 09:55:03,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:03,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:03,235 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=bf230a91f2dd4768974bd854b33e048f, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:03,235 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936503235"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936503235"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936503235"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936503235"}]},"ts":"1675936503235"} 2023-02-09 09:55:03,237 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-02-09 09:55:03,237 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure bf230a91f2dd4768974bd854b33e048f, server=jenkins-hbase4.apache.org,42719,1675936416061 in 170 msec 2023-02-09 09:55:03,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-09 09:55:03,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, REOPEN/MOVE in 1.0960 sec 2023-02-09 09:55:03,240 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-02-09 09:55:03,240 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1000 sec 2023-02-09 09:55:03,241 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5140 sec 2023-02-09 09:55:03,242 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:03,242 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:03,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-02-09 09:55:03,243 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:03,243 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:03,244 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:03,244 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:03,244 INFO [Listener at localhost/42557] compactions.TestMobCompactor(278): alter status finished 2023-02-09 09:55:03,245 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:55:03,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 09:55:03,247 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 09:55:03,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:55:03,248 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937103248 2023-02-09 09:55:03,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:55:03,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-09 09:55:03,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:03,249 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x22c4752d to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:55:03,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@131d8b6c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:55:03,254 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:03,254 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:03,254 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:03,254 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:03,254 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:04,078 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112858386580998648a595063c0b38d1914a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112858386580998648a595063c0b38d1914a 2023-02-09 09:55:04,079 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:04,092 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:55:04,093 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33572, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:55:04,099 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:04,099 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:04,100 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:55:04,101 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41220, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:55:04,109 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4528745395b34aa980db1c29812171ce first=Optional[r05] last=Optional[r8] 2023-02-09 09:55:04,109 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=51 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4528745395b34aa980db1c29812171ce}] 2023-02-09 09:55:04,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4528745395b34aa980db1c29812171ce for inclusion in bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:55:04,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-09 09:55:04,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:55:04,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for bf230a91f2dd4768974bd854b33e048f: 2023-02-09 09:55:04,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4528745395b34aa980db1c29812171ce to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ht93d4rquqt73egmt0knof1rujpj4mksd669vlldi1sdfdi214b6vo9n0lo3a9kj/family1/4528745395b34aa980db1c29812171ce 2023-02-09 09:55:04,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ht93d4rquqt73egmt0knof1rujpj4mksd669vlldi1sdfdi214b6vo9n0lo3a9kj/family1/4528745395b34aa980db1c29812171ce as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/0c11b5cff47d405b9216310293596c58_SeqId_53_ 2023-02-09 09:55:04,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ht93d4rquqt73egmt0knof1rujpj4mksd669vlldi1sdfdi214b6vo9n0lo3a9kj/family1/4528745395b34aa980db1c29812171ce into bf230a91f2dd4768974bd854b33e048f/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/0c11b5cff47d405b9216310293596c58_SeqId_53_ - updating store file list. 2023-02-09 09:55:04,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/0c11b5cff47d405b9216310293596c58_SeqId_53_ into bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:55:04,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ht93d4rquqt73egmt0knof1rujpj4mksd669vlldi1sdfdi214b6vo9n0lo3a9kj/family1/4528745395b34aa980db1c29812171ce into bf230a91f2dd4768974bd854b33e048f/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/0c11b5cff47d405b9216310293596c58_SeqId_53_) 2023-02-09 09:55:04,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ht93d4rquqt73egmt0knof1rujpj4mksd669vlldi1sdfdi214b6vo9n0lo3a9kj/family1/4528745395b34aa980db1c29812171ce 2023-02-09 09:55:04,125 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:04,126 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:04,127 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120e9d8fcfb8e074ae485104570a819ffa9 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120e9d8fcfb8e074ae485104570a819ffa9 2023-02-09 09:55:04,128 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112805121abea9304b7b9ddd776f29b1ade2 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112805121abea9304b7b9ddd776f29b1ade2 2023-02-09 09:55:04,128 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-02-09 09:55:04,128 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 09:55:04,129 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:04,129 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:04,129 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:04,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:04,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:55:04,130 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 09:55:04,131 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 884 msec 2023-02-09 09:55:04,175 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:04,175 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:04,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-02-09 09:55:04,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:55:04,586 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-09 09:55:04,586 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-09 09:55:04,587 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-09 09:55:04,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-09 09:55:04,590 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, REOPEN/MOVE}] 2023-02-09 09:55:04,591 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, REOPEN/MOVE 2023-02-09 09:55:04,592 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=bf230a91f2dd4768974bd854b33e048f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:04,592 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936504591"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936504591"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936504591"}]},"ts":"1675936504591"} 2023-02-09 09:55:04,593 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure bf230a91f2dd4768974bd854b33e048f, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:04,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:04,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bf230a91f2dd4768974bd854b33e048f, disabling compactions & flushes 2023-02-09 09:55:04,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:04,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:04,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. after waiting 0 ms 2023-02-09 09:55:04,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:04,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-09 09:55:04,752 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:04,752 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bf230a91f2dd4768974bd854b33e048f: Waiting for close lock at 1675936504745Running coprocessor pre-close hooks at 1675936504745Disabling compacts and flushes for region at 1675936504745Disabling writes for close at 1675936504745Writing region close event to WAL at 1675936504749 (+4 ms)Running coprocessor post-close hooks at 1675936504752 (+3 ms)Closed at 1675936504752 2023-02-09 09:55:04,752 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: bf230a91f2dd4768974bd854b33e048f to self. 2023-02-09 09:55:04,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:04,753 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=bf230a91f2dd4768974bd854b33e048f, regionState=CLOSED 2023-02-09 09:55:04,753 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936504753"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936504753"}]},"ts":"1675936504753"} 2023-02-09 09:55:04,755 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-02-09 09:55:04,755 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure bf230a91f2dd4768974bd854b33e048f, server=jenkins-hbase4.apache.org,42719,1675936416061 in 161 msec 2023-02-09 09:55:04,756 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=true 2023-02-09 09:55:04,906 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=bf230a91f2dd4768974bd854b33e048f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:04,906 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936504906"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936504906"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936504906"}]},"ts":"1675936504906"} 2023-02-09 09:55:04,908 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure bf230a91f2dd4768974bd854b33e048f, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:05,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:05,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bf230a91f2dd4768974bd854b33e048f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:55:05,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:05,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:05,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:05,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:05,064 INFO [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:05,064 DEBUG [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:55:05,065 DEBUG [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1 2023-02-09 09:55:05,065 INFO [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf230a91f2dd4768974bd854b33e048f columnFamilyName family1 2023-02-09 09:55:05,070 DEBUG [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/0c11b5cff47d405b9216310293596c58_SeqId_53_ 2023-02-09 09:55:05,073 DEBUG [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/family1/8897bb3c6ea14347874dd27118059142 2023-02-09 09:55:05,073 INFO [StoreOpener-bf230a91f2dd4768974bd854b33e048f-1] regionserver.HStore(310): Store=bf230a91f2dd4768974bd854b33e048f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:05,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:05,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:05,077 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:55:05,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bf230a91f2dd4768974bd854b33e048f; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74743020, jitterRate=0.11375778913497925}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:55:05,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bf230a91f2dd4768974bd854b33e048f: Running coprocessor pre-open hook at 1675936505063Writing region info on filesystem at 1675936505063Initializing all the Stores at 1675936505063Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} at 1675936505063Cleaning up temporary data from old regions at 1675936505075 (+12 ms)Cleaning up detritus from prior splits at 1675936505075Running coprocessor post-open hooks at 1675936505077 (+2 ms)Region opened successfully at 1675936505077 2023-02-09 09:55:05,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f., pid=69, masterSystemTime=1675936505059 2023-02-09 09:55:05,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:05,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:55:05,080 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=bf230a91f2dd4768974bd854b33e048f, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:05,080 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675936505080"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936505080"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936505080"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936505080"}]},"ts":"1675936505080"} 2023-02-09 09:55:05,082 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-02-09 09:55:05,082 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure bf230a91f2dd4768974bd854b33e048f, server=jenkins-hbase4.apache.org,42719,1675936416061 in 173 msec 2023-02-09 09:55:05,083 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-02-09 09:55:05,083 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=bf230a91f2dd4768974bd854b33e048f, REOPEN/MOVE in 492 msec 2023-02-09 09:55:05,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-09 09:55:05,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 496 msec 2023-02-09 09:55:05,087 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 910 msec 2023-02-09 09:55:05,087 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:05,087 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:05,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-02-09 09:55:05,088 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:05,088 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:05,089 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:05,089 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:05,089 INFO [Listener at localhost/42557] compactions.TestMobCompactor(278): alter status finished 2023-02-09 09:55:05,090 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 09:55:05,092 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 09:55:05,092 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 09:55:05,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:55:05,093 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937105093 2023-02-09 09:55:05,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:55:05,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-09 09:55:05,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:05,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x73684073 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:55:05,099 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11052ff8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:55:05,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:05,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:05,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:05,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:05,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:05,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:05,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:05,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 09:55:05,100 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 09:55:05,100 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:55:05,101 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 09:55:05,102 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 10 msec 2023-02-09 09:55:05,146 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=240 (was 246), OpenFileDescriptor=621 (was 618) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=200 (was 247), ProcessCount=168 (was 168), AvailableMemoryMB=3357 (was 3401) 2023-02-09 09:55:05,157 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=240, OpenFileDescriptor=621, MaxFileDescriptor=60000, SystemLoadAverage=200, ProcessCount=168, AvailableMemoryMB=3357 2023-02-09 09:55:05,158 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:05,159 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:05,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 09:55:05,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-02-09 09:55:05,162 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:55:05,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-02-09 09:55:05,162 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:05,162 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:05,162 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:55:05,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 09:55:05,163 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:05,164 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82 empty. 2023-02-09 09:55:05,164 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:05,164 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-09 09:55:05,263 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:05,263 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:05,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 09:55:05,464 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:05,464 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:05,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 09:55:05,572 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-02-09 09:55:05,573 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9938c691ec48d26c24056e1121ebcb82, NAME => 'testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:55:05,765 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:05,766 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:05,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 09:55:05,979 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:05,979 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1603): Closing 9938c691ec48d26c24056e1121ebcb82, disabling compactions & flushes 2023-02-09 09:55:05,979 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:05,979 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:05,979 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. after waiting 0 ms 2023-02-09 09:55:05,979 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:05,979 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:05,979 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 9938c691ec48d26c24056e1121ebcb82: Waiting for close lock at 1675936505979Disabling compacts and flushes for region at 1675936505979Disabling writes for close at 1675936505979Writing region close event to WAL at 1675936505979Closed at 1675936505979 2023-02-09 09:55:05,981 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:55:05,982 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675936505982"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936505982"}]},"ts":"1675936505982"} 2023-02-09 09:55:05,983 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 09:55:05,984 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:55:05,984 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936505984"}]},"ts":"1675936505984"} 2023-02-09 09:55:05,985 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-02-09 09:55:05,988 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=9938c691ec48d26c24056e1121ebcb82, ASSIGN}] 2023-02-09 09:55:05,990 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=9938c691ec48d26c24056e1121ebcb82, ASSIGN 2023-02-09 09:55:05,990 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=9938c691ec48d26c24056e1121ebcb82, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:55:06,141 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=9938c691ec48d26c24056e1121ebcb82, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:06,141 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675936506141"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936506141"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936506141"}]},"ts":"1675936506141"} 2023-02-09 09:55:06,143 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 9938c691ec48d26c24056e1121ebcb82, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:06,267 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:06,267 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:06,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 09:55:06,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:06,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9938c691ec48d26c24056e1121ebcb82, NAME => 'testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:55:06,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-02-09 09:55:06,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-02-09 09:55:06,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:06,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:06,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:06,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:06,300 INFO [StoreOpener-9938c691ec48d26c24056e1121ebcb82-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:06,301 DEBUG [StoreOpener-9938c691ec48d26c24056e1121ebcb82-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1 2023-02-09 09:55:06,301 DEBUG [StoreOpener-9938c691ec48d26c24056e1121ebcb82-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1 2023-02-09 09:55:06,302 INFO [StoreOpener-9938c691ec48d26c24056e1121ebcb82-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9938c691ec48d26c24056e1121ebcb82 columnFamilyName f1 2023-02-09 09:55:06,303 INFO [StoreOpener-9938c691ec48d26c24056e1121ebcb82-1] regionserver.HStore(310): Store=9938c691ec48d26c24056e1121ebcb82/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:06,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:06,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:06,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:06,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:55:06,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9938c691ec48d26c24056e1121ebcb82; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72160958, jitterRate=0.07528206706047058}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:55:06,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9938c691ec48d26c24056e1121ebcb82: Running coprocessor pre-open hook at 1675936506299Writing region info on filesystem at 1675936506299Initializing all the Stores at 1675936506300 (+1 ms)Instantiating store for column family {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936506300Cleaning up temporary data from old regions at 1675936506304 (+4 ms)Cleaning up detritus from prior splits at 1675936506305 (+1 ms)Running coprocessor post-open hooks at 1675936506308 (+3 ms)Region opened successfully at 1675936506308 2023-02-09 09:55:06,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82., pid=73, masterSystemTime=1675936506294 2023-02-09 09:55:06,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:06,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:06,310 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=9938c691ec48d26c24056e1121ebcb82, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:06,311 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675936506310"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936506310"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936506310"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936506310"}]},"ts":"1675936506310"} 2023-02-09 09:55:06,313 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-09 09:55:06,313 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 9938c691ec48d26c24056e1121ebcb82, server=jenkins-hbase4.apache.org,42719,1675936416061 in 169 msec 2023-02-09 09:55:06,314 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-02-09 09:55:06,314 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=9938c691ec48d26c24056e1121ebcb82, ASSIGN in 325 msec 2023-02-09 09:55:06,315 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:55:06,315 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936506315"}]},"ts":"1675936506315"} 2023-02-09 09:55:06,316 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-02-09 09:55:06,317 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:55:06,318 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1570 sec 2023-02-09 09:55:07,268 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:07,268 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:07,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 09:55:07,269 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-02-09 09:55:07,269 DEBUG [Listener at localhost/42557] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-02-09 09:55:07,271 INFO [Listener at localhost/42557] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-09 09:55:07,273 INFO [Listener at localhost/42557] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-02-09 09:55:07,274 INFO [Listener at localhost/42557] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-09 09:55:07,274 INFO [Listener at localhost/42557] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-02-09 09:55:07,278 INFO [Listener at localhost/42557] regionserver.HRegion(2744): Flushing 9938c691ec48d26c24056e1121ebcb82 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-09 09:55:07,605 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-02-09 09:55:07,689 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302095b5b6141cb00421fbcc9c1a1425eb076 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302095b5b6141cb00421fbcc9c1a1425eb076 2023-02-09 09:55:08,095 INFO [Listener at localhost/42557] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/.tmp/f1/90fa913db1a74f9f871347a1dfa64e31 2023-02-09 09:55:08,100 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/.tmp/f1/90fa913db1a74f9f871347a1dfa64e31 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31 2023-02-09 09:55:08,103 INFO [Listener at localhost/42557] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31, entries=1, sequenceid=5, filesize=4.9 K 2023-02-09 09:55:08,104 INFO [Listener at localhost/42557] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 9938c691ec48d26c24056e1121ebcb82 in 826ms, sequenceid=5, compaction requested=false 2023-02-09 09:55:08,105 DEBUG [Listener at localhost/42557] regionserver.HRegion(2445): Flush status journal for 9938c691ec48d26c24056e1121ebcb82: 2023-02-09 09:55:08,107 INFO [Listener at localhost/42557] regionserver.HRegion(2744): Flushing 9938c691ec48d26c24056e1121ebcb82 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-09 09:55:08,440 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:55:08,518 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209f996b94d8e02486f83d357b265b83732 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209f996b94d8e02486f83d357b265b83732 2023-02-09 09:55:08,923 INFO [Listener at localhost/42557] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/.tmp/f1/89e54227269b47cb803d705be37ee843 2023-02-09 09:55:08,927 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/.tmp/f1/89e54227269b47cb803d705be37ee843 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843 2023-02-09 09:55:08,931 INFO [Listener at localhost/42557] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843, entries=1, sequenceid=9, filesize=4.9 K 2023-02-09 09:55:08,932 INFO [Listener at localhost/42557] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 9938c691ec48d26c24056e1121ebcb82 in 825ms, sequenceid=9, compaction requested=false 2023-02-09 09:55:08,932 DEBUG [Listener at localhost/42557] regionserver.HRegion(2445): Flush status journal for 9938c691ec48d26c24056e1121ebcb82: 2023-02-09 09:55:08,933 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-02-09 09:55:08,934 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-09 09:55:08,935 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-09 09:55:08,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:55:08,935 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937108935 2023-02-09 09:55:08,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 09:55:08,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-02-09 09:55:08,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-02-09 09:55:08,936 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x12b4e51a to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:55:08,940 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2af6725b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:55:08,940 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testGetAfterCompaction' and column='f1' 2023-02-09 09:55:08,984 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:08,984 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,035 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,035 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,086 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,086 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,137 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,137 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,188 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,188 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,239 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,239 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,289 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,290 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,340 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,340 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,391 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,391 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,442 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,442 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,493 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,493 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,544 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,544 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,595 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,595 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,645 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,646 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,696 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,696 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,747 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,747 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,763 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209ab1b06d047aa438cb5a93c44158f6764 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209ab1b06d047aa438cb5a93c44158f6764 2023-02-09 09:55:09,763 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testGetAfterCompaction' column='f1' 2023-02-09 09:55:09,776 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:55:09,777 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33586, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:55:09,782 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:09,783 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:09,783 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:55:09,785 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41234, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:55:09,792 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/53e263f153da454383cf7b4970ad49b2 first=Optional[r1] last=Optional[r2] 2023-02-09 09:55:09,793 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{f1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/53e263f153da454383cf7b4970ad49b2}] 2023-02-09 09:55:09,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/53e263f153da454383cf7b4970ad49b2 for inclusion in 9938c691ec48d26c24056e1121ebcb82/f1 2023-02-09 09:55:09,798 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,798 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-09 09:55:09,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:55:09,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 9938c691ec48d26c24056e1121ebcb82: 2023-02-09 09:55:09,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/53e263f153da454383cf7b4970ad49b2 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testGetAfterCompaction__3mih9e1vo2rjluraineprem8r6iu1sb0p960g4u6lekre3uqsrv02vg3k6ked7q7/f1/53e263f153da454383cf7b4970ad49b2 2023-02-09 09:55:09,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testGetAfterCompaction__3mih9e1vo2rjluraineprem8r6iu1sb0p960g4u6lekre3uqsrv02vg3k6ked7q7/f1/53e263f153da454383cf7b4970ad49b2 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_ 2023-02-09 09:55:09,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testGetAfterCompaction__3mih9e1vo2rjluraineprem8r6iu1sb0p960g4u6lekre3uqsrv02vg3k6ked7q7/f1/53e263f153da454383cf7b4970ad49b2 into 9938c691ec48d26c24056e1121ebcb82/f1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_ - updating store file list. 2023-02-09 09:55:09,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_ into 9938c691ec48d26c24056e1121ebcb82/f1 2023-02-09 09:55:09,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testGetAfterCompaction__3mih9e1vo2rjluraineprem8r6iu1sb0p960g4u6lekre3uqsrv02vg3k6ked7q7/f1/53e263f153da454383cf7b4970ad49b2 into 9938c691ec48d26c24056e1121ebcb82/f1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_) 2023-02-09 09:55:09,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testGetAfterCompaction__3mih9e1vo2rjluraineprem8r6iu1sb0p960g4u6lekre3uqsrv02vg3k6ked7q7/f1/53e263f153da454383cf7b4970ad49b2 2023-02-09 09:55:09,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(1912): 9938c691ec48d26c24056e1121ebcb82/f1 is initiating minor compaction 2023-02-09 09:55:09,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@51d069c1; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:55:09,806 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9938c691ec48d26c24056e1121ebcb82/f1 in testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:09,806 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/.tmp, totalSize=10.0 K 2023-02-09 09:55:09,806 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 89e54227269b47cb803d705be37ee843, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 09:55:09,806 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 0a8ad8a5da784887a2353fe0548864fb_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 09:55:09,809 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testGetAfterCompaction' column='f1' 2023-02-09 09:55:09,809 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:09,811 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302095b5b6141cb00421fbcc9c1a1425eb076 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302095b5b6141cb00421fbcc9c1a1425eb076 2023-02-09 09:55:09,812 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209f996b94d8e02486f83d357b265b83732 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209f996b94d8e02486f83d357b265b83732 2023-02-09 09:55:09,812 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-02-09 09:55:09,812 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 09:55:09,812 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testGetAfterCompaction' and column='f1' 2023-02-09 09:55:09,812 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-02-09 09:55:09,812 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:55:09,813 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-09 09:55:09,814 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9938c691ec48d26c24056e1121ebcb82#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:09,815 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 879 msec 2023-02-09 09:55:09,849 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:09,849 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,849 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,852 DEBUG [Listener at localhost/42557] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 09:55:09,856 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33592, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 09:55:09,856 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42719] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:09,857 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-02-09 09:55:09,857 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-09 09:55:09,857 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42719] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-09 09:55:09,857 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42719] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. because compaction request was cancelled 2023-02-09 09:55:09,909 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:09,909 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,909 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,960 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:09,960 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:09,960 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,011 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,011 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,011 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,062 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,062 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,062 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,113 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,113 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,113 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,164 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,164 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,164 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,215 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,215 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,215 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,266 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,266 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,266 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,317 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,317 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,317 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,368 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,368 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,368 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,419 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,419 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,419 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,470 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,470 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,470 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,521 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,521 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,521 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,572 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,572 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,572 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,623 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,623 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,623 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,631 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/.tmp/f1/36c5ee7d27a74a07b95922b54d47db2e as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/36c5ee7d27a74a07b95922b54d47db2e 2023-02-09 09:55:10,635 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 9938c691ec48d26c24056e1121ebcb82/f1 of 9938c691ec48d26c24056e1121ebcb82 into 36c5ee7d27a74a07b95922b54d47db2e(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:55:10,635 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9938c691ec48d26c24056e1121ebcb82: 2023-02-09 09:55:10,635 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1675936509806; duration=0sec 2023-02-09 09:55:10,635 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:55:10,674 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,674 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/36c5ee7d27a74a07b95922b54d47db2e; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675936510626; access_time=1675936509812; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,674 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,674 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,725 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675936509759; access_time=1675936508950; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 09:55:10,725 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/36c5ee7d27a74a07b95922b54d47db2e; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675936510626; access_time=1675936509812; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,725 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508922; access_time=1675936508108; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,725 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,727 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_] to archive 2023-02-09 09:55:10,728 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:10,730 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/89e54227269b47cb803d705be37ee843 2023-02-09 09:55:10,731 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/0a8ad8a5da784887a2353fe0548864fb_SeqId_12_ 2023-02-09 09:55:10,776 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/36c5ee7d27a74a07b95922b54d47db2e; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675936510626; access_time=1675936509812; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,776 INFO [Listener at localhost/42557] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675936508094; access_time=1675936507280; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 09:55:10,891 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:10,891 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:10,892 INFO [Listener at localhost/42557] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-02-09 09:55:10,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-02-09 09:55:10,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-02-09 09:55:10,904 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936510904"}]},"ts":"1675936510904"} 2023-02-09 09:55:10,905 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:10,905 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:10,905 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-02-09 09:55:10,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-09 09:55:10,907 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-02-09 09:55:10,908 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=9938c691ec48d26c24056e1121ebcb82, UNASSIGN}] 2023-02-09 09:55:10,909 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=9938c691ec48d26c24056e1121ebcb82, UNASSIGN 2023-02-09 09:55:10,910 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=9938c691ec48d26c24056e1121ebcb82, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:10,910 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675936510910"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936510910"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936510910"}]},"ts":"1675936510910"} 2023-02-09 09:55:10,911 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 9938c691ec48d26c24056e1121ebcb82, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:11,006 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:11,006 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:11,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-09 09:55:11,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:11,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9938c691ec48d26c24056e1121ebcb82, disabling compactions & flushes 2023-02-09 09:55:11,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:11,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:11,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. after waiting 0 ms 2023-02-09 09:55:11,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:11,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-09 09:55:11,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-09 09:55:11,068 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82. 2023-02-09 09:55:11,068 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9938c691ec48d26c24056e1121ebcb82: Waiting for close lock at 1675936511063Running coprocessor pre-close hooks at 1675936511063Disabling compacts and flushes for region at 1675936511063Disabling writes for close at 1675936511063Writing region close event to WAL at 1675936511064 (+1 ms)Running coprocessor post-close hooks at 1675936511067 (+3 ms)Closed at 1675936511068 (+1 ms) 2023-02-09 09:55:11,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:11,070 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=9938c691ec48d26c24056e1121ebcb82, regionState=CLOSED 2023-02-09 09:55:11,070 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675936511070"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936511070"}]},"ts":"1675936511070"} 2023-02-09 09:55:11,072 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-09 09:55:11,072 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 9938c691ec48d26c24056e1121ebcb82, server=jenkins-hbase4.apache.org,42719,1675936416061 in 160 msec 2023-02-09 09:55:11,073 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-02-09 09:55:11,073 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=9938c691ec48d26c24056e1121ebcb82, UNASSIGN in 164 msec 2023-02-09 09:55:11,074 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936511074"}]},"ts":"1675936511074"} 2023-02-09 09:55:11,075 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-02-09 09:55:11,084 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-02-09 09:55:11,085 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 188 msec 2023-02-09 09:55:11,207 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:11,207 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:11,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-09 09:55:11,208 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-02-09 09:55:11,209 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:11,209 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:11,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-02-09 09:55:11,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 09:55:11,220 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 09:55:11,221 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 09:55:11,222 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:11,222 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:11,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-09 09:55:11,224 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:11,226 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1, FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/recovered.edits] 2023-02-09 09:55:11,229 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/36c5ee7d27a74a07b95922b54d47db2e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/36c5ee7d27a74a07b95922b54d47db2e 2023-02-09 09:55:11,230 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/f1/90fa913db1a74f9f871347a1dfa64e31 2023-02-09 09:55:11,232 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/recovered.edits/15.seqid to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82/recovered.edits/15.seqid 2023-02-09 09:55:11,232 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testGetAfterCompaction/9938c691ec48d26c24056e1121ebcb82 2023-02-09 09:55:11,232 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-09 09:55:11,233 DEBUG [PEWorker-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-09 09:55:11,233 DEBUG [PEWorker-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-02-09 09:55:11,236 DEBUG [PEWorker-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209ab1b06d047aa438cb5a93c44158f6764 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209ab1b06d047aa438cb5a93c44158f6764 2023-02-09 09:55:11,237 DEBUG [PEWorker-2] backup.HFileArchiver(596): Deleted hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-09 09:55:11,238 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 09:55:11,258 WARN [PEWorker-2] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-02-09 09:55:11,261 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-02-09 09:55:11,262 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 09:55:11,262 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-02-09 09:55:11,262 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675936511262"}]},"ts":"9223372036854775807"} 2023-02-09 09:55:11,263 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 09:55:11,263 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 9938c691ec48d26c24056e1121ebcb82, NAME => 'testGetAfterCompaction,,1675936505160.9938c691ec48d26c24056e1121ebcb82.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 09:55:11,263 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-02-09 09:55:11,263 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675936511263"}]},"ts":"9223372036854775807"} 2023-02-09 09:55:11,265 INFO [PEWorker-2] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-02-09 09:55:11,266 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 09:55:11,267 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 52 msec 2023-02-09 09:55:11,322 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:11,323 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:11,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-09 09:55:11,324 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-02-09 09:55:11,336 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=245 (was 240) - Thread LEAK? -, OpenFileDescriptor=610 (was 621), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=169 (was 200), ProcessCount=168 (was 168), AvailableMemoryMB=3353 (was 3357) 2023-02-09 09:55:11,347 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=245, OpenFileDescriptor=610, MaxFileDescriptor=60000, SystemLoadAverage=169, ProcessCount=168, AvailableMemoryMB=3353 2023-02-09 09:55:11,348 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:11,348 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:11,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 09:55:11,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-02-09 09:55:11,351 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:55:11,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-02-09 09:55:11,352 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:11,352 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:11,352 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:55:11,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 09:55:11,354 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:11,354 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:11,354 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:11,354 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009 empty. 2023-02-09 09:55:11,354 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9 empty. 2023-02-09 09:55:11,354 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f empty. 2023-02-09 09:55:11,355 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:11,355 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:11,355 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:11,355 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-02-09 09:55:11,453 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:11,453 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:11,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 09:55:11,654 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:11,654 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:11,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 09:55:11,762 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-09 09:55:11,763 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7671): creating {ENCODED => 47fec7fd4b7f8dc5df9e778c5af07aa9, NAME => 'testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:55:11,764 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7671): creating {ENCODED => 60d93642fa0ceecf3be9bd92ffd4e009, NAME => 'testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:55:11,764 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7671): creating {ENCODED => 954cfb87e30cb0d4ca9c9e7b1988704f, NAME => 'testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:55:11,955 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:11,955 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:11,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 09:55:12,174 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:12,174 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1603): Closing 60d93642fa0ceecf3be9bd92ffd4e009, disabling compactions & flushes 2023-02-09 09:55:12,174 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:12,174 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:12,174 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. after waiting 0 ms 2023-02-09 09:55:12,174 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:12,174 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:12,174 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1557): Region close journal for 60d93642fa0ceecf3be9bd92ffd4e009: Waiting for close lock at 1675936512174Disabling compacts and flushes for region at 1675936512174Disabling writes for close at 1675936512174Writing region close event to WAL at 1675936512174Closed at 1675936512174 2023-02-09 09:55:12,178 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:12,178 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1603): Closing 47fec7fd4b7f8dc5df9e778c5af07aa9, disabling compactions & flushes 2023-02-09 09:55:12,178 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:12,178 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:12,178 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. after waiting 0 ms 2023-02-09 09:55:12,178 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:12,178 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:12,178 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1557): Region close journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: Waiting for close lock at 1675936512178Disabling compacts and flushes for region at 1675936512178Disabling writes for close at 1675936512178Writing region close event to WAL at 1675936512178Closed at 1675936512178 2023-02-09 09:55:12,178 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:12,178 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1603): Closing 954cfb87e30cb0d4ca9c9e7b1988704f, disabling compactions & flushes 2023-02-09 09:55:12,179 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:12,179 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:12,179 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. after waiting 0 ms 2023-02-09 09:55:12,179 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:12,179 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:12,179 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1557): Region close journal for 954cfb87e30cb0d4ca9c9e7b1988704f: Waiting for close lock at 1675936512178Disabling compacts and flushes for region at 1675936512178Disabling writes for close at 1675936512179 (+1 ms)Writing region close event to WAL at 1675936512179Closed at 1675936512179 2023-02-09 09:55:12,181 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:55:12,182 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936512181"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936512181"}]},"ts":"1675936512181"} 2023-02-09 09:55:12,182 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936512181"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936512181"}]},"ts":"1675936512181"} 2023-02-09 09:55:12,182 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936512181"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936512181"}]},"ts":"1675936512181"} 2023-02-09 09:55:12,184 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 09:55:12,184 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:55:12,185 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936512184"}]},"ts":"1675936512184"} 2023-02-09 09:55:12,186 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-09 09:55:12,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, ASSIGN}] 2023-02-09 09:55:12,191 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, ASSIGN 2023-02-09 09:55:12,191 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, ASSIGN 2023-02-09 09:55:12,191 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, ASSIGN 2023-02-09 09:55:12,192 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:55:12,192 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:55:12,192 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:55:12,343 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=60d93642fa0ceecf3be9bd92ffd4e009, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:12,343 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=954cfb87e30cb0d4ca9c9e7b1988704f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:12,343 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=47fec7fd4b7f8dc5df9e778c5af07aa9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:12,344 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936512343"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936512343"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936512343"}]},"ts":"1675936512343"} 2023-02-09 09:55:12,344 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936512343"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936512343"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936512343"}]},"ts":"1675936512343"} 2023-02-09 09:55:12,344 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936512343"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936512343"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936512343"}]},"ts":"1675936512343"} 2023-02-09 09:55:12,345 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure 60d93642fa0ceecf3be9bd92ffd4e009, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:12,346 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure 954cfb87e30cb0d4ca9c9e7b1988704f, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:12,347 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=82, state=RUNNABLE; OpenRegionProcedure 47fec7fd4b7f8dc5df9e778c5af07aa9, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:12,456 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:12,456 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:12,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 09:55:12,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:12,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 47fec7fd4b7f8dc5df9e778c5af07aa9, NAME => 'testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 09:55:12,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:12,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:12,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:12,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:12,502 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:12,503 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1 2023-02-09 09:55:12,504 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1 2023-02-09 09:55:12,504 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47fec7fd4b7f8dc5df9e778c5af07aa9 columnFamilyName family1 2023-02-09 09:55:12,505 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(310): Store=47fec7fd4b7f8dc5df9e778c5af07aa9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:12,505 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:12,506 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2 2023-02-09 09:55:12,506 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2 2023-02-09 09:55:12,507 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47fec7fd4b7f8dc5df9e778c5af07aa9 columnFamilyName family2 2023-02-09 09:55:12,508 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(310): Store=47fec7fd4b7f8dc5df9e778c5af07aa9/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:12,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:12,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:12,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:55:12,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:12,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:55:12,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 47fec7fd4b7f8dc5df9e778c5af07aa9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63957123, jitterRate=-0.046964600682258606}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:55:12,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: Running coprocessor pre-open hook at 1675936512501Writing region info on filesystem at 1675936512501Initializing all the Stores at 1675936512502 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936512502Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936512502Cleaning up temporary data from old regions at 1675936512509 (+7 ms)Cleaning up detritus from prior splits at 1675936512510 (+1 ms)Running coprocessor post-open hooks at 1675936512513 (+3 ms)Region opened successfully at 1675936512513 2023-02-09 09:55:12,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9., pid=85, masterSystemTime=1675936512498 2023-02-09 09:55:12,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:12,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:12,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:12,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 954cfb87e30cb0d4ca9c9e7b1988704f, NAME => 'testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 09:55:12,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:12,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:12,516 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=47fec7fd4b7f8dc5df9e778c5af07aa9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:12,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:12,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:12,516 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936512516"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936512516"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936512516"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936512516"}]},"ts":"1675936512516"} 2023-02-09 09:55:12,517 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:12,518 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1 2023-02-09 09:55:12,518 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1 2023-02-09 09:55:12,519 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 954cfb87e30cb0d4ca9c9e7b1988704f columnFamilyName family1 2023-02-09 09:55:12,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=82 2023-02-09 09:55:12,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=82, state=SUCCESS; OpenRegionProcedure 47fec7fd4b7f8dc5df9e778c5af07aa9, server=jenkins-hbase4.apache.org,42719,1675936416061 in 171 msec 2023-02-09 09:55:12,520 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(310): Store=954cfb87e30cb0d4ca9c9e7b1988704f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:12,520 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, ASSIGN in 329 msec 2023-02-09 09:55:12,520 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:12,521 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2 2023-02-09 09:55:12,521 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2 2023-02-09 09:55:12,521 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 954cfb87e30cb0d4ca9c9e7b1988704f columnFamilyName family2 2023-02-09 09:55:12,522 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(310): Store=954cfb87e30cb0d4ca9c9e7b1988704f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:12,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:12,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:12,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:55:12,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:12,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:55:12,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 954cfb87e30cb0d4ca9c9e7b1988704f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67829000, jitterRate=0.010730862617492676}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:55:12,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 954cfb87e30cb0d4ca9c9e7b1988704f: Running coprocessor pre-open hook at 1675936512516Writing region info on filesystem at 1675936512516Initializing all the Stores at 1675936512517 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936512517Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936512517Cleaning up temporary data from old regions at 1675936512524 (+7 ms)Cleaning up detritus from prior splits at 1675936512524Running coprocessor post-open hooks at 1675936512528 (+4 ms)Region opened successfully at 1675936512528 2023-02-09 09:55:12,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f., pid=84, masterSystemTime=1675936512498 2023-02-09 09:55:12,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:12,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:12,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:12,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 60d93642fa0ceecf3be9bd92ffd4e009, NAME => 'testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 09:55:12,530 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=954cfb87e30cb0d4ca9c9e7b1988704f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:12,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:12,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:12,530 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936512530"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936512530"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936512530"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936512530"}]},"ts":"1675936512530"} 2023-02-09 09:55:12,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:12,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:12,531 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:12,532 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1 2023-02-09 09:55:12,532 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1 2023-02-09 09:55:12,532 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-02-09 09:55:12,532 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure 954cfb87e30cb0d4ca9c9e7b1988704f, server=jenkins-hbase4.apache.org,42719,1675936416061 in 185 msec 2023-02-09 09:55:12,532 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60d93642fa0ceecf3be9bd92ffd4e009 columnFamilyName family1 2023-02-09 09:55:12,533 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, ASSIGN in 344 msec 2023-02-09 09:55:12,533 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(310): Store=60d93642fa0ceecf3be9bd92ffd4e009/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:12,534 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:12,535 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2 2023-02-09 09:55:12,535 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2 2023-02-09 09:55:12,535 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60d93642fa0ceecf3be9bd92ffd4e009 columnFamilyName family2 2023-02-09 09:55:12,536 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(310): Store=60d93642fa0ceecf3be9bd92ffd4e009/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:12,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:12,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:12,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:55:12,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:12,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:55:12,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 60d93642fa0ceecf3be9bd92ffd4e009; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64846653, jitterRate=-0.0337095707654953}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:55:12,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 60d93642fa0ceecf3be9bd92ffd4e009: Running coprocessor pre-open hook at 1675936512530Writing region info on filesystem at 1675936512530Initializing all the Stores at 1675936512531 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936512531Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936512531Cleaning up temporary data from old regions at 1675936512537 (+6 ms)Cleaning up detritus from prior splits at 1675936512538 (+1 ms)Running coprocessor post-open hooks at 1675936512541 (+3 ms)Region opened successfully at 1675936512541 2023-02-09 09:55:12,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009., pid=83, masterSystemTime=1675936512498 2023-02-09 09:55:12,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:12,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:12,543 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=60d93642fa0ceecf3be9bd92ffd4e009, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:12,543 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936512543"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936512543"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936512543"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936512543"}]},"ts":"1675936512543"} 2023-02-09 09:55:12,546 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-09 09:55:12,546 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure 60d93642fa0ceecf3be9bd92ffd4e009, server=jenkins-hbase4.apache.org,42719,1675936416061 in 200 msec 2023-02-09 09:55:12,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-09 09:55:12,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, ASSIGN in 358 msec 2023-02-09 09:55:12,547 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:55:12,548 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936512548"}]},"ts":"1675936512548"} 2023-02-09 09:55:12,549 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-09 09:55:12,551 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:55:12,552 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2020 sec 2023-02-09 09:55:13,457 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:13,457 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:13,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 09:55:13,458 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-02-09 09:55:13,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:13,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:13,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:13,470 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:13,470 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:13,470 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:13,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 09:55:13,472 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 09:55:13,472 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:13,472 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 09:55:13,472 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 09:55:13,473 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:13,473 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:13,474 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:13,474 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:13,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:13,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:13,474 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:13,474 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:13,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:13,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:13,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:13,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:13,475 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 09:55:13,475 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 09:55:13,475 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 09:55:13,475 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:13,475 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 09:55:13,475 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:13,476 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:13,476 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:13,476 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. started... 2023-02-09 09:55:13,476 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:55:13,476 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:13,476 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. started... 2023-02-09 09:55:13,476 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. started... 2023-02-09 09:55:13,476 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 954cfb87e30cb0d4ca9c9e7b1988704f 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 09:55:13,477 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 60d93642fa0ceecf3be9bd92ffd4e009 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 09:55:13,477 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 47fec7fd4b7f8dc5df9e778c5af07aa9 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 09:55:13,493 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302093860bd0a392948db8d92023da420e4c7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302093860bd0a392948db8d92023da420e4c7 2023-02-09 09:55:13,494 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209e52b294ef182415eb568aee0413d15f4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209e52b294ef182415eb568aee0413d15f4 2023-02-09 09:55:13,606 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-09 09:55:13,893 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020960b2c1c05c9245d488e8aa1a05b0a7ff to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020960b2c1c05c9245d488e8aa1a05b0a7ff 2023-02-09 09:55:13,897 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/e29bc24dc5194e0f848f9f3cb5264c45 2023-02-09 09:55:13,898 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/e0ed0a83b4f3420b92e9e0b5ff47afa1 2023-02-09 09:55:14,298 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/a6ca269ae6a84d519af27293a70fd223 2023-02-09 09:55:14,311 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302092e33a279f6ac411a934a03f5f911ae76 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302092e33a279f6ac411a934a03f5f911ae76 2023-02-09 09:55:14,312 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209196da5ec53964da7954a31b03f48e88c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209196da5ec53964da7954a31b03f48e88c 2023-02-09 09:55:14,712 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302093fe0ef0f6c924387b73f84ab574e4d47 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302093fe0ef0f6c924387b73f84ab574e4d47 2023-02-09 09:55:14,716 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/983958f5e14940aa8bae33560ee80c22 2023-02-09 09:55:14,717 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/5ddaa897126749fc934e33a744c5d393 2023-02-09 09:55:14,720 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/e29bc24dc5194e0f848f9f3cb5264c45 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/e29bc24dc5194e0f848f9f3cb5264c45 2023-02-09 09:55:14,721 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/e0ed0a83b4f3420b92e9e0b5ff47afa1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/e0ed0a83b4f3420b92e9e0b5ff47afa1 2023-02-09 09:55:14,724 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/e29bc24dc5194e0f848f9f3cb5264c45, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 09:55:14,724 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/e0ed0a83b4f3420b92e9e0b5ff47afa1, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 09:55:14,725 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/983958f5e14940aa8bae33560ee80c22 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/983958f5e14940aa8bae33560ee80c22 2023-02-09 09:55:14,725 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/5ddaa897126749fc934e33a744c5d393 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/5ddaa897126749fc934e33a744c5d393 2023-02-09 09:55:14,729 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/5ddaa897126749fc934e33a744c5d393, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 09:55:14,729 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/983958f5e14940aa8bae33560ee80c22, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 09:55:14,730 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 47fec7fd4b7f8dc5df9e778c5af07aa9 in 1253ms, sequenceid=5, compaction requested=false 2023-02-09 09:55:14,730 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 60d93642fa0ceecf3be9bd92ffd4e009 in 1253ms, sequenceid=5, compaction requested=false 2023-02-09 09:55:14,730 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:14,730 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:14,730 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:14,731 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:14,731 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:55:14,731 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:55:15,117 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/bbbc15242a304f499ce610c7b0bbc247 2023-02-09 09:55:15,121 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/a6ca269ae6a84d519af27293a70fd223 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/a6ca269ae6a84d519af27293a70fd223 2023-02-09 09:55:15,124 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/a6ca269ae6a84d519af27293a70fd223, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 09:55:15,125 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/bbbc15242a304f499ce610c7b0bbc247 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/bbbc15242a304f499ce610c7b0bbc247 2023-02-09 09:55:15,129 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/bbbc15242a304f499ce610c7b0bbc247, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 09:55:15,129 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 954cfb87e30cb0d4ca9c9e7b1988704f in 1653ms, sequenceid=5, compaction requested=false 2023-02-09 09:55:15,130 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:15,130 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:55:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:55:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 09:55:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 09:55:15,131 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:15,131 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:15,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:15,131 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:15,132 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:15,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:15,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:15,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:15,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:15,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 09:55:15,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c759fdd[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:15,133 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 09:55:15,133 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:15,134 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:15,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:15,134 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 09:55:15,134 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 09:55:15,134 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:15,134 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,134 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:15,136 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 09:55:15,136 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:15,136 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,136 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 09:55:15,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:15,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:15,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:15,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:15,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:15,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:15,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:15,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:15,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 09:55:15,138 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 09:55:15,138 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:15,138 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:15,138 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 09:55:15,138 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:15,140 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:15,140 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:15,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:15,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:15,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:15,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:15,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:15,140 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:15,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:15,140 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:15,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:15,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:15,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:15,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:15,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:15,141 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:15,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:15,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,143 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,143 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:15,143 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:15,143 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:15,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:15,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 09:55:15,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:15,143 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:15,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:15,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 09:55:15,143 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:15,143 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,144 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:15,144 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:15,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:15,144 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:15,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:15,144 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:15,144 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:15,244 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:15,244 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:15,244 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:15,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:15,246 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:15,246 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:15,246 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:15,248 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:15,248 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:15,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:15,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:15,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 09:55:15,251 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 09:55:15,251 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:15,251 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 09:55:15,251 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 09:55:15,251 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:15,251 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:15,252 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,252 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:15,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:15,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:15,253 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:15,253 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:15,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:15,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:15,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:15,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:15,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 09:55:15,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 09:55:15,254 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 09:55:15,254 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 09:55:15,254 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:15,254 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 09:55:15,254 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:15,254 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. started... 2023-02-09 09:55:15,254 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 954cfb87e30cb0d4ca9c9e7b1988704f 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 09:55:15,254 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:55:15,254 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:15,254 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:15,254 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. started... 2023-02-09 09:55:15,254 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:15,254 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. started... 2023-02-09 09:55:15,254 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 47fec7fd4b7f8dc5df9e778c5af07aa9 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 09:55:15,254 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 60d93642fa0ceecf3be9bd92ffd4e009 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 09:55:15,265 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020954c62db62733478eaaea1e942c7d0447 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020954c62db62733478eaaea1e942c7d0447 2023-02-09 09:55:15,666 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209d3b01fffc2b84619a402d32d19edf15d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209d3b01fffc2b84619a402d32d19edf15d 2023-02-09 09:55:15,666 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302099ccb9095ff214264a5e3aaf03df096ff to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302099ccb9095ff214264a5e3aaf03df096ff 2023-02-09 09:55:15,669 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/0e819bfb780044c193e7afcbfd576d56 2023-02-09 09:55:16,071 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/cf96736150a04e7dafa94513af336383 2023-02-09 09:55:16,071 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/2b4c5e1a5d624abda847254ed690fee2 2023-02-09 09:55:16,082 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209b1a803e2ec6747ed9d71da7de9db7d33 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209b1a803e2ec6747ed9d71da7de9db7d33 2023-02-09 09:55:16,486 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209f2cf67b8d6e04a4a8852cb9a7cefca35 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209f2cf67b8d6e04a4a8852cb9a7cefca35 2023-02-09 09:55:16,486 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209373b70eeab504282afa2bb41ac264825 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209373b70eeab504282afa2bb41ac264825 2023-02-09 09:55:16,487 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/06ce406e16c7488c8aeccab2d5d8b934 2023-02-09 09:55:16,491 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/0e819bfb780044c193e7afcbfd576d56 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/0e819bfb780044c193e7afcbfd576d56 2023-02-09 09:55:16,495 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/0e819bfb780044c193e7afcbfd576d56, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 09:55:16,495 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/06ce406e16c7488c8aeccab2d5d8b934 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/06ce406e16c7488c8aeccab2d5d8b934 2023-02-09 09:55:16,498 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/06ce406e16c7488c8aeccab2d5d8b934, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 09:55:16,499 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 60d93642fa0ceecf3be9bd92ffd4e009 in 1245ms, sequenceid=9, compaction requested=false 2023-02-09 09:55:16,499 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:16,499 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:16,499 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:55:16,570 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:55:16,891 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/f12746f2a255473696f11886211e7782 2023-02-09 09:55:16,892 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/ee5c35bfe19f4a0fa91bada511c58d14 2023-02-09 09:55:16,896 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/cf96736150a04e7dafa94513af336383 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/cf96736150a04e7dafa94513af336383 2023-02-09 09:55:16,896 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/2b4c5e1a5d624abda847254ed690fee2 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/2b4c5e1a5d624abda847254ed690fee2 2023-02-09 09:55:16,899 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/cf96736150a04e7dafa94513af336383, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 09:55:16,900 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/2b4c5e1a5d624abda847254ed690fee2, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 09:55:16,900 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/f12746f2a255473696f11886211e7782 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/f12746f2a255473696f11886211e7782 2023-02-09 09:55:16,900 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/ee5c35bfe19f4a0fa91bada511c58d14 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/ee5c35bfe19f4a0fa91bada511c58d14 2023-02-09 09:55:16,904 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/f12746f2a255473696f11886211e7782, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 09:55:16,905 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 47fec7fd4b7f8dc5df9e778c5af07aa9 in 1651ms, sequenceid=9, compaction requested=false 2023-02-09 09:55:16,905 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:16,905 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:16,905 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:55:16,906 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/ee5c35bfe19f4a0fa91bada511c58d14, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 09:55:16,906 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 954cfb87e30cb0d4ca9c9e7b1988704f in 1652ms, sequenceid=9, compaction requested=false 2023-02-09 09:55:16,906 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:16,906 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:16,906 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:55:16,907 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:55:16,907 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,907 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 09:55:16,907 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 09:55:16,909 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:16,909 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:16,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:16,909 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:16,909 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:16,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:16,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:16,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:16,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:16,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 09:55:16,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1737c39d[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:16,910 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 09:55:16,911 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:16,911 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:16,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:16,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:16,912 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 09:55:16,912 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 09:55:16,912 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:16,912 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,912 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:16,913 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 09:55:16,913 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,913 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:16,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,913 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 09:55:16,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:16,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:16,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:16,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:16,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:16,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:16,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:16,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:16,915 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 09:55:16,915 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 09:55:16,915 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:16,915 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:16,915 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 09:55:16,915 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:16,916 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:16,916 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:16,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:16,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:16,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:16,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:16,916 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:16,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:16,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:16,917 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:16,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:16,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:16,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:16,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:16,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:16,917 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:16,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:16,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,920 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,920 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:16,920 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:16,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:16,920 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:16,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 09:55:16,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 09:55:16,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:16,920 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:16,920 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:16,920 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:16,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:16,920 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:16,921 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:16,921 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:16,921 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:16,921 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:16,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:16,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:17,021 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:17,021 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:17,021 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:17,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:17,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:17,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:17,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:17,025 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:17,026 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:17,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:17,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:17,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 09:55:17,028 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 09:55:17,028 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:17,028 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 09:55:17,028 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 09:55:17,028 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:17,028 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:17,029 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:17,029 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:17,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:17,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:17,030 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:17,030 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:17,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:17,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:17,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:17,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:17,030 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 09:55:17,030 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 09:55:17,030 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 09:55:17,031 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 09:55:17,031 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:17,031 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 09:55:17,031 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:17,031 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. started... 2023-02-09 09:55:17,031 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:17,031 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 954cfb87e30cb0d4ca9c9e7b1988704f 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 09:55:17,031 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:17,031 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. started... 2023-02-09 09:55:17,031 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:55:17,031 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:17,031 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 47fec7fd4b7f8dc5df9e778c5af07aa9 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 09:55:17,031 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. started... 2023-02-09 09:55:17,031 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 60d93642fa0ceecf3be9bd92ffd4e009 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 09:55:17,056 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209ea061cd2e689491fb9eaadae01f0b4e0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ea061cd2e689491fb9eaadae01f0b4e0 2023-02-09 09:55:17,060 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/96179ee988fa437280604c295c8ed11c 2023-02-09 09:55:17,456 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302099a8f83e7795d44e39f56953e4a247cad to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302099a8f83e7795d44e39f56953e4a247cad 2023-02-09 09:55:17,457 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020986b92434695045c8b7fd2993f6273d25 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020986b92434695045c8b7fd2993f6273d25 2023-02-09 09:55:17,472 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209ebbc78182e394898a21e1e6118635fba to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209ebbc78182e394898a21e1e6118635fba 2023-02-09 09:55:17,864 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/593f9755d5f34a68b34e9e24cb4620c0 2023-02-09 09:55:17,864 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/76b31550dba244f68b84fc182e34921b 2023-02-09 09:55:17,880 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/9f82b2ac0f6948f293091af9dd3f0ddc 2023-02-09 09:55:17,884 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/96179ee988fa437280604c295c8ed11c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/96179ee988fa437280604c295c8ed11c 2023-02-09 09:55:17,887 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/96179ee988fa437280604c295c8ed11c, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 09:55:17,888 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/9f82b2ac0f6948f293091af9dd3f0ddc as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f82b2ac0f6948f293091af9dd3f0ddc 2023-02-09 09:55:17,890 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f82b2ac0f6948f293091af9dd3f0ddc, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 09:55:17,891 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 60d93642fa0ceecf3be9bd92ffd4e009 in 860ms, sequenceid=13, compaction requested=true 2023-02-09 09:55:17,891 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:17,891 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:55:18,281 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302096ed54f417c6d4539b8912e8428a3387d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302096ed54f417c6d4539b8912e8428a3387d 2023-02-09 09:55:18,284 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020953c0b8a3fc3a46f9bf1579619e7167b7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020953c0b8a3fc3a46f9bf1579619e7167b7 2023-02-09 09:55:18,687 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/1bb332c62cde4076a782e66eda402986 2023-02-09 09:55:18,689 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/70bee5b32c144a35876afcc27e2e38b0 2023-02-09 09:55:18,691 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/593f9755d5f34a68b34e9e24cb4620c0 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/593f9755d5f34a68b34e9e24cb4620c0 2023-02-09 09:55:18,694 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/76b31550dba244f68b84fc182e34921b as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/76b31550dba244f68b84fc182e34921b 2023-02-09 09:55:18,696 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/593f9755d5f34a68b34e9e24cb4620c0, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 09:55:18,697 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/1bb332c62cde4076a782e66eda402986 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/1bb332c62cde4076a782e66eda402986 2023-02-09 09:55:18,698 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/76b31550dba244f68b84fc182e34921b, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 09:55:18,699 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/70bee5b32c144a35876afcc27e2e38b0 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/70bee5b32c144a35876afcc27e2e38b0 2023-02-09 09:55:18,703 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/1bb332c62cde4076a782e66eda402986, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 09:55:18,704 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 954cfb87e30cb0d4ca9c9e7b1988704f in 1673ms, sequenceid=13, compaction requested=true 2023-02-09 09:55:18,704 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:18,704 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:18,704 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:55:18,705 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/70bee5b32c144a35876afcc27e2e38b0, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 09:55:18,705 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 47fec7fd4b7f8dc5df9e778c5af07aa9 in 1674ms, sequenceid=13, compaction requested=true 2023-02-09 09:55:18,706 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:18,706 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:18,706 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:55:18,706 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:55:18,706 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,706 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 09:55:18,706 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 09:55:18,707 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,707 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:18,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:18,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:18,708 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:18,708 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:18,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:18,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:18,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:18,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:18,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 09:55:18,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e3d051[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:18,709 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 09:55:18,709 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:18,710 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:18,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:18,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:18,710 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 09:55:18,710 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 09:55:18,710 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:18,710 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,710 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:18,711 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 09:55:18,711 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:18,711 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,711 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 09:55:18,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:18,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:18,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:18,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:18,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:18,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:18,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:18,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:18,713 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 09:55:18,713 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 09:55:18,713 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:18,713 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:18,713 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 09:55:18,713 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:18,718 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:18,718 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:18,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:18,718 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:18,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:18,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:18,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:18,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:18,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:18,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:18,719 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:18,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:18,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:18,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:18,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:18,719 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:18,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:18,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,721 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:18,721 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:18,721 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:18,721 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:18,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:18,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 09:55:18,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:18,721 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:18,722 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:18,722 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 09:55:18,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:18,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:18,722 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:18,722 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:18,722 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:18,722 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:18,722 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:18,822 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:18,822 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:18,822 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:18,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:18,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:18,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:18,825 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:18,826 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:18,826 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:18,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:18,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:18,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 09:55:18,829 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 09:55:18,829 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:18,829 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 09:55:18,829 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 09:55:18,829 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:18,829 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:18,830 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:18,830 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:18,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:18,831 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:18,831 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:18,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:18,831 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:18,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:18,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:18,831 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 09:55:18,832 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 09:55:18,832 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 09:55:18,832 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 09:55:18,832 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:18,832 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 09:55:18,832 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:18,832 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. started... 2023-02-09 09:55:18,832 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:55:18,832 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:18,832 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:18,832 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 954cfb87e30cb0d4ca9c9e7b1988704f 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 09:55:18,832 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:18,832 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. started... 2023-02-09 09:55:18,832 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. started... 2023-02-09 09:55:18,832 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 47fec7fd4b7f8dc5df9e778c5af07aa9 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 09:55:18,832 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 60d93642fa0ceecf3be9bd92ffd4e009 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 09:55:19,254 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020939be72032efb49678cbf53e19763c1d3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020939be72032efb49678cbf53e19763c1d3 2023-02-09 09:55:19,255 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302094e5d780559a742c0b6d327a4c8921b48 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302094e5d780559a742c0b6d327a4c8921b48 2023-02-09 09:55:19,255 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209a2bade1a5de94df69caf1593d3311949 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209a2bade1a5de94df69caf1593d3311949 2023-02-09 09:55:19,662 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/a93265fde28c461eaaa4e09ead22cce4 2023-02-09 09:55:19,665 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/4e9bb78e0c574fd186f7760fae94a250 2023-02-09 09:55:19,666 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/ff516574d54647dd87b46ba263d32631 2023-02-09 09:55:20,076 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020906ce2eb8cbc34ce3b7f7d5586c0714d8 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020906ce2eb8cbc34ce3b7f7d5586c0714d8 2023-02-09 09:55:20,083 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302098e6acf6524e243f48bbaee4a9d740e3c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302098e6acf6524e243f48bbaee4a9d740e3c 2023-02-09 09:55:20,088 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302095a0868e46faa47a5a907d4db4962f3ea to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302095a0868e46faa47a5a907d4db4962f3ea 2023-02-09 09:55:20,481 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/9f08785f19784b10858a6bbf3398a180 2023-02-09 09:55:20,485 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/a93265fde28c461eaaa4e09ead22cce4 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/a93265fde28c461eaaa4e09ead22cce4 2023-02-09 09:55:20,487 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/23e13d2980384fa29816e32999b316ec 2023-02-09 09:55:20,489 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/a93265fde28c461eaaa4e09ead22cce4, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 09:55:20,490 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/9f08785f19784b10858a6bbf3398a180 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f08785f19784b10858a6bbf3398a180 2023-02-09 09:55:20,491 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/4e9bb78e0c574fd186f7760fae94a250 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/4e9bb78e0c574fd186f7760fae94a250 2023-02-09 09:55:20,492 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/838362fd9450402e84f7f7a8eddd6562 2023-02-09 09:55:20,493 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f08785f19784b10858a6bbf3398a180, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 09:55:20,494 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/4e9bb78e0c574fd186f7760fae94a250, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 09:55:20,495 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/23e13d2980384fa29816e32999b316ec as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/23e13d2980384fa29816e32999b316ec 2023-02-09 09:55:20,496 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 60d93642fa0ceecf3be9bd92ffd4e009 in 1664ms, sequenceid=17, compaction requested=true 2023-02-09 09:55:20,496 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:20,496 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:20,496 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:55:20,497 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/ff516574d54647dd87b46ba263d32631 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/ff516574d54647dd87b46ba263d32631 2023-02-09 09:55:20,499 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/23e13d2980384fa29816e32999b316ec, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 09:55:20,499 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 954cfb87e30cb0d4ca9c9e7b1988704f in 1667ms, sequenceid=17, compaction requested=true 2023-02-09 09:55:20,499 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:20,500 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:20,500 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:55:20,500 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/ff516574d54647dd87b46ba263d32631, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 09:55:20,501 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/838362fd9450402e84f7f7a8eddd6562 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/838362fd9450402e84f7f7a8eddd6562 2023-02-09 09:55:20,504 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/838362fd9450402e84f7f7a8eddd6562, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 09:55:20,505 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 47fec7fd4b7f8dc5df9e778c5af07aa9 in 1673ms, sequenceid=17, compaction requested=true 2023-02-09 09:55:20,505 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:20,505 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:20,505 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:55:20,505 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:55:20,505 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,505 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 09:55:20,505 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 09:55:20,506 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:20,506 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:20,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:20,507 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:20,507 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:20,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:20,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:20,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:20,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:20,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 09:55:20,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e7ad982[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:20,508 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 09:55:20,508 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:20,510 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:20,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:20,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:20,510 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 09:55:20,510 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 09:55:20,510 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:20,510 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,510 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:20,511 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 09:55:20,511 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,511 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:20,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:20,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:20,511 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 09:55:20,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:20,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:20,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:20,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:20,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:20,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:20,513 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 09:55:20,513 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:20,513 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:20,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 09:55:20,513 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 09:55:20,513 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:20,514 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:20,514 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:20,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:20,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:20,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:20,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:20,515 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:20,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:20,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:20,515 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:20,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:20,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:20,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:20,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:20,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:20,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,516 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:20,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:20,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,518 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,518 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:20,518 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:20,518 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:20,518 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:20,518 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:20,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 09:55:20,518 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:20,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:20,518 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:20,518 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:20,519 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:20,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:20,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:20,519 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:20,519 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:20,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:20,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:20,619 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:20,619 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:20,619 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:20,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:20,629 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:20,629 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:20,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 09:55:20,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1675936520630 (current time:1675936520630). 2023-02-09 09:55:20,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-09 09:55:20,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] zookeeper.ReadOnlyZKClient(139): Connect 0x74336940 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:55:20,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@758b4b40, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:55:20,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:55:20,644 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39918, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:55:20,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x74336940 to 127.0.0.1:64170 2023-02-09 09:55:20,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:55:20,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-09 09:55:20,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 09:55:20,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 09:55:20,654 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 09:55:20,655 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1675937120655 2023-02-09 09:55:20,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 09:55:20,655 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1675936520620 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-09 09:55:20,657 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 09:55:20,657 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-09 09:55:20,657 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 4 msec 2023-02-09 09:55:20,658 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-09 09:55:20,658 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1675937120658 2023-02-09 09:55:20,661 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:20,661 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-09 09:55:20,761 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 09:55:20,762 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:20,762 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:20,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 09:55:20,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 09:55:20,765 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-09 09:55:20,965 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 09:55:20,965 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:20,966 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:20,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 09:55:20,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 09:55:20,966 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-09 09:55:21,072 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1675936520620 2023-02-09 09:55:21,072 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1675936520620' 2023-02-09 09:55:21,072 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-09 09:55:21,072 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1675936520620' starting 'acquire' 2023-02-09 09:55:21,072 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1675936520620', kicking off acquire phase on members. 2023-02-09 09:55:21,072 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1675936520620 2023-02-09 09:55:21,072 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1675936520620 2023-02-09 09:55:21,073 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-09 09:55:21,073 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-09 09:55:21,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 09:55:21,074 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,074 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:21,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1675936520620 2023-02-09 09:55:21,074 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1675936520620 2023-02-09 09:55:21,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-02-09 09:55:21,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1675936520620 2023-02-09 09:55:21,075 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1675936520620 from table testCompactionWithHFileLink type FLUSH 2023-02-09 09:55:21,077 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1675936520620 2023-02-09 09:55:21,077 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1675936520620' with timeout 300000ms 2023-02-09 09:55:21,077 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-09 09:55:21,077 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1675936520620' starting 'acquire' stage 2023-02-09 09:55:21,077 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1675936520620' locally acquired 2023-02-09 09:55:21,077 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (snaptb-1675936520620) in zk 2023-02-09 09:55:21,078 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,078 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1675936520620 2023-02-09 09:55:21,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:21,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-09 09:55:21,078 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1675936520620 2023-02-09 09:55:21,078 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1675936520620' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:21,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:21,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:21,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675936520620 2023-02-09 09:55:21,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:21,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'snaptb-1675936520620' on coordinator 2023-02-09 09:55:21,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ef13ae7[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:21,080 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1675936520620' starting 'in-barrier' execution. 2023-02-09 09:55:21,080 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1675936520620 2023-02-09 09:55:21,082 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675936520620 2023-02-09 09:55:21,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1675936520620 2023-02-09 09:55:21,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1675936520620 2023-02-09 09:55:21,082 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1675936520620' received 'reached' from coordinator. 2023-02-09 09:55:21,082 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,082 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:21,083 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:21,083 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:21,083 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-02-09 09:55:21,083 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:21,083 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-02-09 09:55:21,083 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. started... 2023-02-09 09:55:21,083 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. started... 2023-02-09 09:55:21,083 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. started... 2023-02-09 09:55:21,084 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] regionserver.HRegion(2445): Flush status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:21,084 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] regionserver.HRegion(2445): Flush status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:21,084 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] regionserver.HRegion(2445): Flush status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:21,084 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.' region-info for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,084 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.' region-info for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,084 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.' region-info for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,089 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-09 09:55:21,089 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-09 09:55:21,089 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-09 09:55:21,093 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/a6ca269ae6a84d519af27293a70fd223, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/2b4c5e1a5d624abda847254ed690fee2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/593f9755d5f34a68b34e9e24cb4620c0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/4e9bb78e0c574fd186f7760fae94a250] hfiles 2023-02-09 09:55:21,093 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/e29bc24dc5194e0f848f9f3cb5264c45, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/cf96736150a04e7dafa94513af336383, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/76b31550dba244f68b84fc182e34921b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/ff516574d54647dd87b46ba263d32631] hfiles 2023-02-09 09:55:21,093 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/e0ed0a83b4f3420b92e9e0b5ff47afa1, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/0e819bfb780044c193e7afcbfd576d56, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/96179ee988fa437280604c295c8ed11c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/a93265fde28c461eaaa4e09ead22cce4] hfiles 2023-02-09 09:55:21,093 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/e29bc24dc5194e0f848f9f3cb5264c45 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,093 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/a6ca269ae6a84d519af27293a70fd223 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,093 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/e0ed0a83b4f3420b92e9e0b5ff47afa1 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/2b4c5e1a5d624abda847254ed690fee2 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/cf96736150a04e7dafa94513af336383 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/0e819bfb780044c193e7afcbfd576d56 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/76b31550dba244f68b84fc182e34921b for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/593f9755d5f34a68b34e9e24cb4620c0 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/ff516574d54647dd87b46ba263d32631 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/4e9bb78e0c574fd186f7760fae94a250 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/96179ee988fa437280604c295c8ed11c for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/a93265fde28c461eaaa4e09ead22cce4 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/bbbc15242a304f499ce610c7b0bbc247, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/ee5c35bfe19f4a0fa91bada511c58d14, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/1bb332c62cde4076a782e66eda402986, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/23e13d2980384fa29816e32999b316ec] hfiles 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/5ddaa897126749fc934e33a744c5d393, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/06ce406e16c7488c8aeccab2d5d8b934, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f82b2ac0f6948f293091af9dd3f0ddc, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f08785f19784b10858a6bbf3398a180] hfiles 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/5ddaa897126749fc934e33a744c5d393 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/bbbc15242a304f499ce610c7b0bbc247 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/983958f5e14940aa8bae33560ee80c22, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/f12746f2a255473696f11886211e7782, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/70bee5b32c144a35876afcc27e2e38b0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/838362fd9450402e84f7f7a8eddd6562] hfiles 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/ee5c35bfe19f4a0fa91bada511c58d14 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/06ce406e16c7488c8aeccab2d5d8b934 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/1bb332c62cde4076a782e66eda402986 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f82b2ac0f6948f293091af9dd3f0ddc for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/983958f5e14940aa8bae33560ee80c22 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f08785f19784b10858a6bbf3398a180 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/f12746f2a255473696f11886211e7782 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/23e13d2980384fa29816e32999b316ec for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/70bee5b32c144a35876afcc27e2e38b0 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,097 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/838362fd9450402e84f7f7a8eddd6562 for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,102 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. completed. 2023-02-09 09:55:21,103 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:21,103 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. completed. 2023-02-09 09:55:21,103 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:21,103 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-02-09 09:55:21,103 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-02-09 09:55:21,267 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 09:55:21,267 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:21,267 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:21,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 09:55:21,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 09:55:21,268 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-09 09:55:21,503 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. completed. 2023-02-09 09:55:21,503 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:21,503 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-02-09 09:55:21,503 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-02-09 09:55:21,503 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,503 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1675936520620' locally completed 2023-02-09 09:55:21,504 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1675936520620' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:21,505 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1675936520620' has notified controller of completion 2023-02-09 09:55:21,505 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:21,505 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,505 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1675936520620' completed. 2023-02-09 09:55:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-09 09:55:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:21,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:21,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675936520620 2023-02-09 09:55:21,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:21,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675936520620 2023-02-09 09:55:21,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1675936520620' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:21,507 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'snaptb-1675936520620', counting down latch. Waiting for 0 more 2023-02-09 09:55:21,507 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1675936520620' execution completed 2023-02-09 09:55:21,507 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:21,507 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:21,507 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1675936520620 2023-02-09 09:55:21,507 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1675936520620including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-02-09 09:55:21,509 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675936520620 2023-02-09 09:55:21,509 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675936520620 2023-02-09 09:55:21,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1675936520620 2023-02-09 09:55:21,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1675936520620 2023-02-09 09:55:21,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1675936520620 2023-02-09 09:55:21,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:21,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-09 09:55:21,509 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-09 09:55:21,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-09 09:55:21,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 09:55:21,509 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:21,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1675936520620 2023-02-09 09:55:21,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675936520620 2023-02-09 09:55:21,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:21,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675936520620 2023-02-09 09:55:21,510 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:21,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675936520620 2023-02-09 09:55:21,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,512 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,512 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-09 09:55:21,512 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675936520620 2023-02-09 09:55:21,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-09 09:55:21,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 09:55:21,512 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1675936520620 2023-02-09 09:55:21,512 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-09 09:55:21,512 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675936520620 2023-02-09 09:55:21,512 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-02-09 09:55:21,512 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:21,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-09 09:55:21,512 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675936520620/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:21,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 09:55:21,512 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1675936520620 2023-02-09 09:55:21,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1675936520620 because node does not exist (not an error) 2023-02-09 09:55:21,513 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-02-09 09:55:21,512 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675936520620 2023-02-09 09:55:21,513 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675936520620 2023-02-09 09:55:21,513 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675936520620 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020960b2c1c05c9245d488e8aa1a05b0a7ff, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302099a8f83e7795d44e39f56953e4a247cad, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302099ccb9095ff214264a5e3aaf03df096ff, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209a2bade1a5de94df69caf1593d3311949, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302093860bd0a392948db8d92023da420e4c7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020939be72032efb49678cbf53e19763c1d3, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020986b92434695045c8b7fd2993f6273d25, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209d3b01fffc2b84619a402d32d19edf15d, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302094e5d780559a742c0b6d327a4c8921b48, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020954c62db62733478eaaea1e942c7d0447, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209e52b294ef182415eb568aee0413d15f4, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ea061cd2e689491fb9eaadae01f0b4e0] mob files 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020960b2c1c05c9245d488e8aa1a05b0a7ff 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302099a8f83e7795d44e39f56953e4a247cad 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302099ccb9095ff214264a5e3aaf03df096ff 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209a2bade1a5de94df69caf1593d3311949 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302093860bd0a392948db8d92023da420e4c7 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020939be72032efb49678cbf53e19763c1d3 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020986b92434695045c8b7fd2993f6273d25 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209d3b01fffc2b84619a402d32d19edf15d 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302094e5d780559a742c0b6d327a4c8921b48 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020954c62db62733478eaaea1e942c7d0447 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209e52b294ef182415eb568aee0413d15f4 2023-02-09 09:55:21,514 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ea061cd2e689491fb9eaadae01f0b4e0 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209373b70eeab504282afa2bb41ac264825, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302093fe0ef0f6c924387b73f84ab574e4d47, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302096ed54f417c6d4539b8912e8428a3387d, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302098e6acf6524e243f48bbaee4a9d740e3c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302092e33a279f6ac411a934a03f5f911ae76, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020953c0b8a3fc3a46f9bf1579619e7167b7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302095a0868e46faa47a5a907d4db4962f3ea, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209f2cf67b8d6e04a4a8852cb9a7cefca35, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020906ce2eb8cbc34ce3b7f7d5586c0714d8, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209196da5ec53964da7954a31b03f48e88c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209b1a803e2ec6747ed9d71da7de9db7d33, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209ebbc78182e394898a21e1e6118635fba] mob files 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209373b70eeab504282afa2bb41ac264825 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302093fe0ef0f6c924387b73f84ab574e4d47 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302096ed54f417c6d4539b8912e8428a3387d 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302098e6acf6524e243f48bbaee4a9d740e3c 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302092e33a279f6ac411a934a03f5f911ae76 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020953c0b8a3fc3a46f9bf1579619e7167b7 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302095a0868e46faa47a5a907d4db4962f3ea 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209f2cf67b8d6e04a4a8852cb9a7cefca35 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020906ce2eb8cbc34ce3b7f7d5586c0714d8 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209196da5ec53964da7954a31b03f48e88c 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209b1a803e2ec6747ed9d71da7de9db7d33 2023-02-09 09:55:21,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209ebbc78182e394898a21e1e6118635fba 2023-02-09 09:55:21,768 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 09:55:21,768 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:21,768 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:21,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 09:55:21,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 09:55:21,769 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-09 09:55:21,920 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1675936520620 2023-02-09 09:55:21,922 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/.tmp/snaptb-1675936520620 2023-02-09 09:55:22,365 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/.tmp/snaptb-1675936520620 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/snaptb-1675936520620 2023-02-09 09:55:22,379 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1675936520620 of table testCompactionWithHFileLink completed 2023-02-09 09:55:22,379 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/.tmp/snaptb-1675936520620 2023-02-09 09:55:22,379 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/.tmp/snaptb-1675936520620 2023-02-09 09:55:22,379 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1675936520620 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1675936520655Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1675936521920 (+1265 ms)Consolidate snapshot: snaptb-1675936520620 at 1675936521920Loading Region manifests for snaptb-1675936520620 at 1675936521920Writing data manifest for snaptb-1675936520620 at 1675936521930 (+10 ms)Verifying snapshot: snaptb-1675936520620 at 1675936522338 (+408 ms)Snapshot snaptb-1675936520620 of table testCompactionWithHFileLink completed at 1675936522379 (+41 ms) 2023-02-09 09:55:22,380 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-09 09:55:22,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.7230 sec 2023-02-09 09:55:22,769 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 09:55:22,769 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:22,769 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:22,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 09:55:22,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-09 09:55:22,796 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:22,797 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:22,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:22,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:22,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 09:55:22,799 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 09:55:22,799 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:22,799 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 09:55:22,799 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 09:55:22,799 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:22,799 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:22,802 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:22,802 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:22,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:22,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:22,802 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:22,802 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:22,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:22,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:22,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:22,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:22,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 09:55:22,803 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 09:55:22,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 09:55:22,803 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 09:55:22,803 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:22,803 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 09:55:22,803 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:22,803 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:55:22,803 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:22,803 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:22,803 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. started... 2023-02-09 09:55:22,803 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:22,803 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. started... 2023-02-09 09:55:22,803 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 60d93642fa0ceecf3be9bd92ffd4e009 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 09:55:22,803 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. started... 2023-02-09 09:55:22,804 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 954cfb87e30cb0d4ca9c9e7b1988704f 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 09:55:22,804 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 47fec7fd4b7f8dc5df9e778c5af07aa9 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 09:55:23,618 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/892fa5263ab546a1b3d07079c0993988 2023-02-09 09:55:23,618 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/2676cd6f59a24936aa59b0a5091f2fd9 2023-02-09 09:55:23,618 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/c02f32cea91646aaac8684fba2b2ad7c 2023-02-09 09:55:23,622 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 892fa5263ab546a1b3d07079c0993988 2023-02-09 09:55:23,622 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2676cd6f59a24936aa59b0a5091f2fd9 2023-02-09 09:55:23,622 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c02f32cea91646aaac8684fba2b2ad7c 2023-02-09 09:55:24,436 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/241c03101ae44e2ba5a6559eb6686132 2023-02-09 09:55:24,437 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/d4e9dfb2698b45a0a0010912136f72a6 2023-02-09 09:55:24,440 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4e9dfb2698b45a0a0010912136f72a6 2023-02-09 09:55:24,440 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 241c03101ae44e2ba5a6559eb6686132 2023-02-09 09:55:24,440 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/eb9a057606314ed69cfad80b73b55d19 2023-02-09 09:55:24,441 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/2676cd6f59a24936aa59b0a5091f2fd9 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/2676cd6f59a24936aa59b0a5091f2fd9 2023-02-09 09:55:24,441 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/c02f32cea91646aaac8684fba2b2ad7c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/c02f32cea91646aaac8684fba2b2ad7c 2023-02-09 09:55:24,444 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb9a057606314ed69cfad80b73b55d19 2023-02-09 09:55:24,444 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/892fa5263ab546a1b3d07079c0993988 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/892fa5263ab546a1b3d07079c0993988 2023-02-09 09:55:24,445 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c02f32cea91646aaac8684fba2b2ad7c 2023-02-09 09:55:24,445 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2676cd6f59a24936aa59b0a5091f2fd9 2023-02-09 09:55:24,445 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/c02f32cea91646aaac8684fba2b2ad7c, entries=3, sequenceid=24, filesize=5.0 K 2023-02-09 09:55:24,445 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/2676cd6f59a24936aa59b0a5091f2fd9, entries=3, sequenceid=24, filesize=5.0 K 2023-02-09 09:55:24,446 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/241c03101ae44e2ba5a6559eb6686132 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/241c03101ae44e2ba5a6559eb6686132 2023-02-09 09:55:24,446 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/d4e9dfb2698b45a0a0010912136f72a6 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/d4e9dfb2698b45a0a0010912136f72a6 2023-02-09 09:55:24,469 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x73684073 to 127.0.0.1:64170 2023-02-09 09:55:24,469 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:55:24,469 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:55:24,469 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x12b4e51a to 127.0.0.1:64170 2023-02-09 09:55:24,469 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:55:24,469 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:55:24,469 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x22c4752d to 127.0.0.1:64170 2023-02-09 09:55:24,470 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:55:24,470 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:55:24,470 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1db827c7 to 127.0.0.1:64170 2023-02-09 09:55:24,470 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:55:24,471 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 892fa5263ab546a1b3d07079c0993988 2023-02-09 09:55:24,471 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/892fa5263ab546a1b3d07079c0993988, entries=3, sequenceid=24, filesize=5.0 K 2023-02-09 09:55:24,472 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 241c03101ae44e2ba5a6559eb6686132 2023-02-09 09:55:24,472 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/241c03101ae44e2ba5a6559eb6686132, entries=1, sequenceid=24, filesize=5.0 K 2023-02-09 09:55:24,473 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4e9dfb2698b45a0a0010912136f72a6 2023-02-09 09:55:24,473 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/d4e9dfb2698b45a0a0010912136f72a6, entries=1, sequenceid=24, filesize=5.0 K 2023-02-09 09:55:24,476 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 954cfb87e30cb0d4ca9c9e7b1988704f in 1672ms, sequenceid=24, compaction requested=true 2023-02-09 09:55:24,476 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 60d93642fa0ceecf3be9bd92ffd4e009 in 1673ms, sequenceid=24, compaction requested=true 2023-02-09 09:55:24,476 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:24,476 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:24,476 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:24,477 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/eb9a057606314ed69cfad80b73b55d19 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/eb9a057606314ed69cfad80b73b55d19 2023-02-09 09:55:24,477 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:24,477 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:55:24,477 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:55:24,481 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb9a057606314ed69cfad80b73b55d19 2023-02-09 09:55:24,481 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/eb9a057606314ed69cfad80b73b55d19, entries=1, sequenceid=24, filesize=5.0 K 2023-02-09 09:55:24,482 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 47fec7fd4b7f8dc5df9e778c5af07aa9 in 1678ms, sequenceid=24, compaction requested=true 2023-02-09 09:55:24,482 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:24,482 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:24,482 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:55:24,482 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:55:24,482 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,482 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 09:55:24,482 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 09:55:24,672 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,672 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:24,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:24,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:24,673 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:24,673 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:24,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:24,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:24,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:24,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:24,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 09:55:24,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57f3258f[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:24,674 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 09:55:24,674 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:24,683 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:24,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:24,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:24,683 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 09:55:24,683 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 09:55:24,683 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:24,684 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,684 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:24,685 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 09:55:24,685 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:24,685 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,685 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 09:55:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:24,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 09:55:24,687 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 09:55:24,687 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:24,687 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:24,687 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 09:55:24,687 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:24,688 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:24,688 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:24,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:24,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:24,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:24,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:24,688 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:24,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:24,688 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:24,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:24,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:24,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:24,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:24,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:24,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:24,689 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:24,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 09:55:24,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,691 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,691 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:24,691 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:24,691 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:24,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:24,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 09:55:24,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:24,691 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:24,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:24,692 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 09:55:24,692 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:24,692 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:24,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 09:55:24,692 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 09:55:24,692 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:24,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:24,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:24,692 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 09:55:24,692 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:24,792 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:24,792 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:24,793 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:24,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:24,794 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:24,794 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): 954cfb87e30cb0d4ca9c9e7b1988704f/family1 is initiating major compaction (all files) 2023-02-09 09:55:24,794 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:24,794 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:24,795 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of 954cfb87e30cb0d4ca9c9e7b1988704f/family1 in testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:24,795 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/a6ca269ae6a84d519af27293a70fd223, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/2b4c5e1a5d624abda847254ed690fee2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/593f9755d5f34a68b34e9e24cb4620c0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/4e9bb78e0c574fd186f7760fae94a250, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/c02f32cea91646aaac8684fba2b2ad7c] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp, totalSize=26.7 K 2023-02-09 09:55:24,796 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting a6ca269ae6a84d519af27293a70fd223, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513467 2023-02-09 09:55:24,796 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 2b4c5e1a5d624abda847254ed690fee2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:24,796 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 593f9755d5f34a68b34e9e24cb4620c0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517024 2023-02-09 09:55:24,797 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 4e9bb78e0c574fd186f7760fae94a250, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518824 2023-02-09 09:55:24,797 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting c02f32cea91646aaac8684fba2b2ad7c, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 09:55:24,809 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): 954cfb87e30cb0d4ca9c9e7b1988704f#family1#compaction#107 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:25,217 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209aba457113c49475488f4cb7106b4cb67_del 2023-02-09 09:55:25,217 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209aba457113c49475488f4cb7106b4cb67_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209aba457113c49475488f4cb7106b4cb67_del 2023-02-09 09:55:26,030 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cebe03bad444a69912eafa88e28763c 2023-02-09 09:55:26,031 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/0cebe03bad444a69912eafa88e28763c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/0cebe03bad444a69912eafa88e28763c 2023-02-09 09:55:26,034 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cebe03bad444a69912eafa88e28763c 2023-02-09 09:55:26,035 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 954cfb87e30cb0d4ca9c9e7b1988704f/family1 of 954cfb87e30cb0d4ca9c9e7b1988704f into 0cebe03bad444a69912eafa88e28763c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:26,035 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:26,035 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:26,035 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): 954cfb87e30cb0d4ca9c9e7b1988704f/family2 is initiating major compaction (all files) 2023-02-09 09:55:26,035 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:26,035 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:26,036 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of 954cfb87e30cb0d4ca9c9e7b1988704f/family2 in testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:26,036 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/bbbc15242a304f499ce610c7b0bbc247, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/ee5c35bfe19f4a0fa91bada511c58d14, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/1bb332c62cde4076a782e66eda402986, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/23e13d2980384fa29816e32999b316ec, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/241c03101ae44e2ba5a6559eb6686132] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp, totalSize=25.4 K 2023-02-09 09:55:26,036 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting bbbc15242a304f499ce610c7b0bbc247, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513467 2023-02-09 09:55:26,037 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting ee5c35bfe19f4a0fa91bada511c58d14, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:26,037 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 1bb332c62cde4076a782e66eda402986, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517024 2023-02-09 09:55:26,037 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 23e13d2980384fa29816e32999b316ec, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518824 2023-02-09 09:55:26,038 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 241c03101ae44e2ba5a6559eb6686132, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 09:55:26,049 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): 954cfb87e30cb0d4ca9c9e7b1988704f#family2#compaction#108 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:26,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/a6ca269ae6a84d519af27293a70fd223, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/2b4c5e1a5d624abda847254ed690fee2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/593f9755d5f34a68b34e9e24cb4620c0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/4e9bb78e0c574fd186f7760fae94a250, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/c02f32cea91646aaac8684fba2b2ad7c] to archive 2023-02-09 09:55:26,128 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:26,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/a6ca269ae6a84d519af27293a70fd223 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/a6ca269ae6a84d519af27293a70fd223 2023-02-09 09:55:26,130 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/2b4c5e1a5d624abda847254ed690fee2 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/2b4c5e1a5d624abda847254ed690fee2 2023-02-09 09:55:26,131 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/593f9755d5f34a68b34e9e24cb4620c0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/593f9755d5f34a68b34e9e24cb4620c0 2023-02-09 09:55:26,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/4e9bb78e0c574fd186f7760fae94a250 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/4e9bb78e0c574fd186f7760fae94a250 2023-02-09 09:55:26,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/c02f32cea91646aaac8684fba2b2ad7c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/c02f32cea91646aaac8684fba2b2ad7c 2023-02-09 09:55:26,457 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209425f6e2534b24126a8a724fcf420bcf9_del 2023-02-09 09:55:26,457 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209425f6e2534b24126a8a724fcf420bcf9_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209425f6e2534b24126a8a724fcf420bcf9_del 2023-02-09 09:55:27,270 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0897d8ff023c4c7ca70bab0eaf928a61 2023-02-09 09:55:27,270 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/0897d8ff023c4c7ca70bab0eaf928a61 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/0897d8ff023c4c7ca70bab0eaf928a61 2023-02-09 09:55:27,273 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0897d8ff023c4c7ca70bab0eaf928a61 2023-02-09 09:55:27,274 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 954cfb87e30cb0d4ca9c9e7b1988704f/family2 of 954cfb87e30cb0d4ca9c9e7b1988704f into 0897d8ff023c4c7ca70bab0eaf928a61(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:27,274 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:27,274 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:27,274 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 is initiating major compaction (all files) 2023-02-09 09:55:27,274 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:27,275 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:27,275 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 in testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:27,275 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/e29bc24dc5194e0f848f9f3cb5264c45, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/cf96736150a04e7dafa94513af336383, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/76b31550dba244f68b84fc182e34921b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/ff516574d54647dd87b46ba263d32631, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/892fa5263ab546a1b3d07079c0993988] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp, totalSize=26.7 K 2023-02-09 09:55:27,276 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting e29bc24dc5194e0f848f9f3cb5264c45, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513467 2023-02-09 09:55:27,276 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting cf96736150a04e7dafa94513af336383, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:27,276 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 76b31550dba244f68b84fc182e34921b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517024 2023-02-09 09:55:27,276 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting ff516574d54647dd87b46ba263d32631, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518825 2023-02-09 09:55:27,277 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 892fa5263ab546a1b3d07079c0993988, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 09:55:27,287 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): 47fec7fd4b7f8dc5df9e778c5af07aa9#family1#compaction#109 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:27,294 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020995e5a9b60f5f459bb8b030b02f683434_del 2023-02-09 09:55:27,295 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020995e5a9b60f5f459bb8b030b02f683434_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020995e5a9b60f5f459bb8b030b02f683434_del 2023-02-09 09:55:27,327 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/bbbc15242a304f499ce610c7b0bbc247, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/ee5c35bfe19f4a0fa91bada511c58d14, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/1bb332c62cde4076a782e66eda402986, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/23e13d2980384fa29816e32999b316ec, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/241c03101ae44e2ba5a6559eb6686132] to archive 2023-02-09 09:55:27,328 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:27,329 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/bbbc15242a304f499ce610c7b0bbc247 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/bbbc15242a304f499ce610c7b0bbc247 2023-02-09 09:55:27,330 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/ee5c35bfe19f4a0fa91bada511c58d14 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/ee5c35bfe19f4a0fa91bada511c58d14 2023-02-09 09:55:27,331 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/1bb332c62cde4076a782e66eda402986 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/1bb332c62cde4076a782e66eda402986 2023-02-09 09:55:27,332 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/23e13d2980384fa29816e32999b316ec to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/23e13d2980384fa29816e32999b316ec 2023-02-09 09:55:27,333 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/241c03101ae44e2ba5a6559eb6686132 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/241c03101ae44e2ba5a6559eb6686132 2023-02-09 09:55:28,110 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0eefc6df16b64194b2b7dfa903aca90e 2023-02-09 09:55:28,111 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/0eefc6df16b64194b2b7dfa903aca90e as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/0eefc6df16b64194b2b7dfa903aca90e 2023-02-09 09:55:28,115 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0eefc6df16b64194b2b7dfa903aca90e 2023-02-09 09:55:28,116 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 of 47fec7fd4b7f8dc5df9e778c5af07aa9 into 0eefc6df16b64194b2b7dfa903aca90e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:55:28,116 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:28,116 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:28,116 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): 47fec7fd4b7f8dc5df9e778c5af07aa9/family2 is initiating major compaction (all files) 2023-02-09 09:55:28,116 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:28,116 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:28,116 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of 47fec7fd4b7f8dc5df9e778c5af07aa9/family2 in testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:28,116 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/983958f5e14940aa8bae33560ee80c22, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/f12746f2a255473696f11886211e7782, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/70bee5b32c144a35876afcc27e2e38b0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/838362fd9450402e84f7f7a8eddd6562, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/eb9a057606314ed69cfad80b73b55d19] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp, totalSize=25.4 K 2023-02-09 09:55:28,117 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 983958f5e14940aa8bae33560ee80c22, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513467 2023-02-09 09:55:28,117 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting f12746f2a255473696f11886211e7782, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:28,117 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 70bee5b32c144a35876afcc27e2e38b0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517024 2023-02-09 09:55:28,118 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 838362fd9450402e84f7f7a8eddd6562, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518825 2023-02-09 09:55:28,118 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting eb9a057606314ed69cfad80b73b55d19, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 09:55:28,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/e29bc24dc5194e0f848f9f3cb5264c45, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/cf96736150a04e7dafa94513af336383, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/76b31550dba244f68b84fc182e34921b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/ff516574d54647dd87b46ba263d32631, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/892fa5263ab546a1b3d07079c0993988] to archive 2023-02-09 09:55:28,128 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:28,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/e29bc24dc5194e0f848f9f3cb5264c45 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/e29bc24dc5194e0f848f9f3cb5264c45 2023-02-09 09:55:28,129 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): 47fec7fd4b7f8dc5df9e778c5af07aa9#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:28,130 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/cf96736150a04e7dafa94513af336383 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/cf96736150a04e7dafa94513af336383 2023-02-09 09:55:28,131 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/76b31550dba244f68b84fc182e34921b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/76b31550dba244f68b84fc182e34921b 2023-02-09 09:55:28,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/ff516574d54647dd87b46ba263d32631 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/ff516574d54647dd87b46ba263d32631 2023-02-09 09:55:28,133 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/892fa5263ab546a1b3d07079c0993988 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/892fa5263ab546a1b3d07079c0993988 2023-02-09 09:55:28,538 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302095d6070c7e4a04255b03e462902cab150_del 2023-02-09 09:55:28,538 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302095d6070c7e4a04255b03e462902cab150_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302095d6070c7e4a04255b03e462902cab150_del 2023-02-09 09:55:29,351 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1f3fb59308a46df8f99c72d392adde3 2023-02-09 09:55:29,352 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/c1f3fb59308a46df8f99c72d392adde3 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/c1f3fb59308a46df8f99c72d392adde3 2023-02-09 09:55:29,355 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1f3fb59308a46df8f99c72d392adde3 2023-02-09 09:55:29,356 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 47fec7fd4b7f8dc5df9e778c5af07aa9/family2 of 47fec7fd4b7f8dc5df9e778c5af07aa9 into c1f3fb59308a46df8f99c72d392adde3(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:29,356 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:29,356 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:29,356 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): 60d93642fa0ceecf3be9bd92ffd4e009/family1 is initiating major compaction (all files) 2023-02-09 09:55:29,356 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:29,357 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:29,357 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of 60d93642fa0ceecf3be9bd92ffd4e009/family1 in testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:29,357 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/e0ed0a83b4f3420b92e9e0b5ff47afa1, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/0e819bfb780044c193e7afcbfd576d56, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/96179ee988fa437280604c295c8ed11c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/a93265fde28c461eaaa4e09ead22cce4, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/2676cd6f59a24936aa59b0a5091f2fd9] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp, totalSize=26.7 K 2023-02-09 09:55:29,358 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting e0ed0a83b4f3420b92e9e0b5ff47afa1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513465 2023-02-09 09:55:29,358 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 0e819bfb780044c193e7afcbfd576d56, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:29,358 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 96179ee988fa437280604c295c8ed11c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517023 2023-02-09 09:55:29,358 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting a93265fde28c461eaaa4e09ead22cce4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518824 2023-02-09 09:55:29,359 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 2676cd6f59a24936aa59b0a5091f2fd9, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 09:55:29,368 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): 60d93642fa0ceecf3be9bd92ffd4e009#family1#compaction#111 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:29,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/983958f5e14940aa8bae33560ee80c22, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/f12746f2a255473696f11886211e7782, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/70bee5b32c144a35876afcc27e2e38b0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/838362fd9450402e84f7f7a8eddd6562, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/eb9a057606314ed69cfad80b73b55d19] to archive 2023-02-09 09:55:29,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:29,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/983958f5e14940aa8bae33560ee80c22 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/983958f5e14940aa8bae33560ee80c22 2023-02-09 09:55:29,434 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/f12746f2a255473696f11886211e7782 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/f12746f2a255473696f11886211e7782 2023-02-09 09:55:29,435 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/70bee5b32c144a35876afcc27e2e38b0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/70bee5b32c144a35876afcc27e2e38b0 2023-02-09 09:55:29,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/838362fd9450402e84f7f7a8eddd6562 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/838362fd9450402e84f7f7a8eddd6562 2023-02-09 09:55:29,437 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/eb9a057606314ed69cfad80b73b55d19 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/eb9a057606314ed69cfad80b73b55d19 2023-02-09 09:55:29,775 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209111bd4598f5b4668a43fa7e0fbfa36ce_del 2023-02-09 09:55:29,775 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209111bd4598f5b4668a43fa7e0fbfa36ce_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209111bd4598f5b4668a43fa7e0fbfa36ce_del 2023-02-09 09:55:30,588 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ce5b4c408e84375aa69ddcb6aece814 2023-02-09 09:55:30,589 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/6ce5b4c408e84375aa69ddcb6aece814 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/6ce5b4c408e84375aa69ddcb6aece814 2023-02-09 09:55:30,593 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ce5b4c408e84375aa69ddcb6aece814 2023-02-09 09:55:30,594 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 60d93642fa0ceecf3be9bd92ffd4e009/family1 of 60d93642fa0ceecf3be9bd92ffd4e009 into 6ce5b4c408e84375aa69ddcb6aece814(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:30,594 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:30,594 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:30,594 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): 60d93642fa0ceecf3be9bd92ffd4e009/family2 is initiating major compaction (all files) 2023-02-09 09:55:30,595 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:30,595 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:30,595 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of 60d93642fa0ceecf3be9bd92ffd4e009/family2 in testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:30,595 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/5ddaa897126749fc934e33a744c5d393, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/06ce406e16c7488c8aeccab2d5d8b934, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f82b2ac0f6948f293091af9dd3f0ddc, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f08785f19784b10858a6bbf3398a180, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/d4e9dfb2698b45a0a0010912136f72a6] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp, totalSize=25.4 K 2023-02-09 09:55:30,596 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 5ddaa897126749fc934e33a744c5d393, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513465 2023-02-09 09:55:30,596 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 06ce406e16c7488c8aeccab2d5d8b934, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:30,596 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 9f82b2ac0f6948f293091af9dd3f0ddc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517023 2023-02-09 09:55:30,597 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 9f08785f19784b10858a6bbf3398a180, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518824 2023-02-09 09:55:30,597 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting d4e9dfb2698b45a0a0010912136f72a6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 09:55:30,608 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): 60d93642fa0ceecf3be9bd92ffd4e009#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:30,628 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/e0ed0a83b4f3420b92e9e0b5ff47afa1, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/0e819bfb780044c193e7afcbfd576d56, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/96179ee988fa437280604c295c8ed11c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/a93265fde28c461eaaa4e09ead22cce4, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/2676cd6f59a24936aa59b0a5091f2fd9] to archive 2023-02-09 09:55:30,629 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:30,630 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/e0ed0a83b4f3420b92e9e0b5ff47afa1 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/e0ed0a83b4f3420b92e9e0b5ff47afa1 2023-02-09 09:55:30,631 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/0e819bfb780044c193e7afcbfd576d56 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/0e819bfb780044c193e7afcbfd576d56 2023-02-09 09:55:30,633 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/96179ee988fa437280604c295c8ed11c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/96179ee988fa437280604c295c8ed11c 2023-02-09 09:55:30,633 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/a93265fde28c461eaaa4e09ead22cce4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/a93265fde28c461eaaa4e09ead22cce4 2023-02-09 09:55:30,634 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/2676cd6f59a24936aa59b0a5091f2fd9 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/2676cd6f59a24936aa59b0a5091f2fd9 2023-02-09 09:55:31,015 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209c6ef9ffacce24f6c80b5190072283490_del 2023-02-09 09:55:31,015 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209c6ef9ffacce24f6c80b5190072283490_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209c6ef9ffacce24f6c80b5190072283490_del 2023-02-09 09:55:31,829 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a32261a786554d2191915906d58bb23b 2023-02-09 09:55:31,830 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/a32261a786554d2191915906d58bb23b as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/a32261a786554d2191915906d58bb23b 2023-02-09 09:55:31,833 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a32261a786554d2191915906d58bb23b 2023-02-09 09:55:31,834 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 60d93642fa0ceecf3be9bd92ffd4e009/family2 of 60d93642fa0ceecf3be9bd92ffd4e009 into a32261a786554d2191915906d58bb23b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:31,834 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:31,838 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:31,842 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:31,846 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:31,847 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:31,853 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 09:55:31,887 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-09 09:55:31,891 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209111bd4598f5b4668a43fa7e0fbfa36ce_del 2023-02-09 09:55:31,894 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209aba457113c49475488f4cb7106b4cb67_del 2023-02-09 09:55:31,897 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020995e5a9b60f5f459bb8b030b02f683434_del 2023-02-09 09:55:31,897 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:31,897 DEBUG [Listener at localhost/42557] zookeeper.ReadOnlyZKClient(139): Connect 0x0869b1f4 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:55:31,902 DEBUG [Listener at localhost/42557] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65196d35, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:55:31,902 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:31,902 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:31,903 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:31,929 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/5ddaa897126749fc934e33a744c5d393, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/06ce406e16c7488c8aeccab2d5d8b934, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f82b2ac0f6948f293091af9dd3f0ddc, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f08785f19784b10858a6bbf3398a180, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/d4e9dfb2698b45a0a0010912136f72a6] to archive 2023-02-09 09:55:31,931 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:31,936 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/5ddaa897126749fc934e33a744c5d393 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/5ddaa897126749fc934e33a744c5d393 2023-02-09 09:55:31,938 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/06ce406e16c7488c8aeccab2d5d8b934 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/06ce406e16c7488c8aeccab2d5d8b934 2023-02-09 09:55:31,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f82b2ac0f6948f293091af9dd3f0ddc to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f82b2ac0f6948f293091af9dd3f0ddc 2023-02-09 09:55:31,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f08785f19784b10858a6bbf3398a180 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/9f08785f19784b10858a6bbf3398a180 2023-02-09 09:55:31,944 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/d4e9dfb2698b45a0a0010912136f72a6 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/d4e9dfb2698b45a0a0010912136f72a6 2023-02-09 09:55:32,474 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-02-09 09:55:32,883 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209989e11dde2e64d2984f01aa92f3931c9 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209989e11dde2e64d2984f01aa92f3931c9 2023-02-09 09:55:32,883 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020909f30b5701ba4bccb7cd0c30d72be1fb to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020909f30b5701ba4bccb7cd0c30d72be1fb 2023-02-09 09:55:32,883 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209644d989a6911435aa486eb60cafc4702 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209644d989a6911435aa486eb60cafc4702 2023-02-09 09:55:32,884 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:32,884 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:32,884 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:32,899 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:55:32,903 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37282, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:55:32,909 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:32,909 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:32,909 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:32,909 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:32,910 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:55:32,912 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40518, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:55:32,912 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:32,912 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:32,912 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:32,912 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:32,922 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/0bc9f91bf9844ac5bc87a34a4a8e2191 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-09 09:55:32,922 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/43e82e7091c84aebb5a12b07f28d2227 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-09 09:55:32,924 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/0bc9f91bf9844ac5bc87a34a4a8e2191}] 2023-02-09 09:55:32,924 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/be2a4e564c0141fbacaab37bc2bd0b92 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-09 09:55:32,924 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/43e82e7091c84aebb5a12b07f28d2227}] 2023-02-09 09:55:32,925 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/be2a4e564c0141fbacaab37bc2bd0b92}] 2023-02-09 09:55:32,929 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/0bc9f91bf9844ac5bc87a34a4a8e2191 for inclusion in 954cfb87e30cb0d4ca9c9e7b1988704f/family1 2023-02-09 09:55:32,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/be2a4e564c0141fbacaab37bc2bd0b92 for inclusion in 60d93642fa0ceecf3be9bd92ffd4e009/family1 2023-02-09 09:55:32,929 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/43e82e7091c84aebb5a12b07f28d2227 for inclusion in 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 2023-02-09 09:55:32,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-09 09:55:32,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-09 09:55:32,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 09:55:32,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 09:55:32,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:32,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:32,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-09 09:55:32,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 09:55:32,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:32,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/43e82e7091c84aebb5a12b07f28d2227 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__dhr4fk9fsnvd1jqpruihkdn4ij1o2o1afbmj4gndv58tei8mdqei7mvrv9eo6m1i/family1/43e82e7091c84aebb5a12b07f28d2227 2023-02-09 09:55:32,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/be2a4e564c0141fbacaab37bc2bd0b92 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__oj5qamkbea5685nsag3ivojn60enlfb9jo32kcqe6hdd7gn0v08klsgle9tegeu4/family1/be2a4e564c0141fbacaab37bc2bd0b92 2023-02-09 09:55:32,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/0bc9f91bf9844ac5bc87a34a4a8e2191 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__rcav0oug1jpugtke6g37l8fe5steg2hi28qaaripdv7uh23o1q60pkfdqlu1362r/family1/0bc9f91bf9844ac5bc87a34a4a8e2191 2023-02-09 09:55:32,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__rcav0oug1jpugtke6g37l8fe5steg2hi28qaaripdv7uh23o1q60pkfdqlu1362r/family1/0bc9f91bf9844ac5bc87a34a4a8e2191 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/d0e4a898f1d44d9994a293fedf182eff_SeqId_29_ 2023-02-09 09:55:32,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__dhr4fk9fsnvd1jqpruihkdn4ij1o2o1afbmj4gndv58tei8mdqei7mvrv9eo6m1i/family1/43e82e7091c84aebb5a12b07f28d2227 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/81e9ecae595346538da1437c3c4fe864_SeqId_29_ 2023-02-09 09:55:32,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__oj5qamkbea5685nsag3ivojn60enlfb9jo32kcqe6hdd7gn0v08klsgle9tegeu4/family1/be2a4e564c0141fbacaab37bc2bd0b92 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/67390f8a971b4505b0c1d98ca7801392_SeqId_29_ 2023-02-09 09:55:32,935 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__rcav0oug1jpugtke6g37l8fe5steg2hi28qaaripdv7uh23o1q60pkfdqlu1362r/family1/0bc9f91bf9844ac5bc87a34a4a8e2191 into 954cfb87e30cb0d4ca9c9e7b1988704f/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/d0e4a898f1d44d9994a293fedf182eff_SeqId_29_ - updating store file list. 2023-02-09 09:55:32,935 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__dhr4fk9fsnvd1jqpruihkdn4ij1o2o1afbmj4gndv58tei8mdqei7mvrv9eo6m1i/family1/43e82e7091c84aebb5a12b07f28d2227 into 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/81e9ecae595346538da1437c3c4fe864_SeqId_29_ - updating store file list. 2023-02-09 09:55:32,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__oj5qamkbea5685nsag3ivojn60enlfb9jo32kcqe6hdd7gn0v08klsgle9tegeu4/family1/be2a4e564c0141fbacaab37bc2bd0b92 into 60d93642fa0ceecf3be9bd92ffd4e009/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/67390f8a971b4505b0c1d98ca7801392_SeqId_29_ - updating store file list. 2023-02-09 09:55:32,945 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/81e9ecae595346538da1437c3c4fe864_SeqId_29_ into 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 2023-02-09 09:55:32,945 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__dhr4fk9fsnvd1jqpruihkdn4ij1o2o1afbmj4gndv58tei8mdqei7mvrv9eo6m1i/family1/43e82e7091c84aebb5a12b07f28d2227 into 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/81e9ecae595346538da1437c3c4fe864_SeqId_29_) 2023-02-09 09:55:32,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__dhr4fk9fsnvd1jqpruihkdn4ij1o2o1afbmj4gndv58tei8mdqei7mvrv9eo6m1i/family1/43e82e7091c84aebb5a12b07f28d2227 2023-02-09 09:55:32,945 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/d0e4a898f1d44d9994a293fedf182eff_SeqId_29_ into 954cfb87e30cb0d4ca9c9e7b1988704f/family1 2023-02-09 09:55:32,946 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__rcav0oug1jpugtke6g37l8fe5steg2hi28qaaripdv7uh23o1q60pkfdqlu1362r/family1/0bc9f91bf9844ac5bc87a34a4a8e2191 into 954cfb87e30cb0d4ca9c9e7b1988704f/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/d0e4a898f1d44d9994a293fedf182eff_SeqId_29_) 2023-02-09 09:55:32,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/67390f8a971b4505b0c1d98ca7801392_SeqId_29_ into 60d93642fa0ceecf3be9bd92ffd4e009/family1 2023-02-09 09:55:32,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__oj5qamkbea5685nsag3ivojn60enlfb9jo32kcqe6hdd7gn0v08klsgle9tegeu4/family1/be2a4e564c0141fbacaab37bc2bd0b92 into 60d93642fa0ceecf3be9bd92ffd4e009/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/67390f8a971b4505b0c1d98ca7801392_SeqId_29_) 2023-02-09 09:55:32,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__rcav0oug1jpugtke6g37l8fe5steg2hi28qaaripdv7uh23o1q60pkfdqlu1362r/family1/0bc9f91bf9844ac5bc87a34a4a8e2191 2023-02-09 09:55:32,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__oj5qamkbea5685nsag3ivojn60enlfb9jo32kcqe6hdd7gn0v08klsgle9tegeu4/family1/be2a4e564c0141fbacaab37bc2bd0b92 2023-02-09 09:55:32,948 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:32,948 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:32,948 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:32,948 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:32,949 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:32,949 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:32,950 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020960b2c1c05c9245d488e8aa1a05b0a7ff to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020960b2c1c05c9245d488e8aa1a05b0a7ff 2023-02-09 09:55:32,950 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302093860bd0a392948db8d92023da420e4c7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302093860bd0a392948db8d92023da420e4c7 2023-02-09 09:55:32,950 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302094e5d780559a742c0b6d327a4c8921b48 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302094e5d780559a742c0b6d327a4c8921b48 2023-02-09 09:55:32,951 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302099a8f83e7795d44e39f56953e4a247cad to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302099a8f83e7795d44e39f56953e4a247cad 2023-02-09 09:55:32,951 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020939be72032efb49678cbf53e19763c1d3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020939be72032efb49678cbf53e19763c1d3 2023-02-09 09:55:32,951 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020954c62db62733478eaaea1e942c7d0447 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020954c62db62733478eaaea1e942c7d0447 2023-02-09 09:55:32,952 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302099ccb9095ff214264a5e3aaf03df096ff to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302099ccb9095ff214264a5e3aaf03df096ff 2023-02-09 09:55:32,952 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020986b92434695045c8b7fd2993f6273d25 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020986b92434695045c8b7fd2993f6273d25 2023-02-09 09:55:32,952 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209e52b294ef182415eb568aee0413d15f4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209e52b294ef182415eb568aee0413d15f4 2023-02-09 09:55:32,953 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209a2bade1a5de94df69caf1593d3311949 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209a2bade1a5de94df69caf1593d3311949 2023-02-09 09:55:32,953 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 09:55:32,953 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 09:55:32,953 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209d3b01fffc2b84619a402d32d19edf15d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209d3b01fffc2b84619a402d32d19edf15d 2023-02-09 09:55:32,953 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 09:55:32,953 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 09:55:32,953 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ea061cd2e689491fb9eaadae01f0b4e0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ea061cd2e689491fb9eaadae01f0b4e0 2023-02-09 09:55:32,954 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 09:55:32,954 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 09:55:32,954 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:32,954 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:32,954 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:32,954 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:32,954 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:32,954 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209111bd4598f5b4668a43fa7e0fbfa36ce_del] 2023-02-09 09:55:32,955 DEBUG [Listener at localhost/42557] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:32,956 DEBUG [Listener at localhost/42557] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209111bd4598f5b4668a43fa7e0fbfa36ce_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209111bd4598f5b4668a43fa7e0fbfa36ce_del 2023-02-09 09:55:32,956 INFO [Listener at localhost/42557] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-09 09:55:32,956 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209aba457113c49475488f4cb7106b4cb67_del] 2023-02-09 09:55:32,956 DEBUG [Listener at localhost/42557] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:32,957 DEBUG [Listener at localhost/42557] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209aba457113c49475488f4cb7106b4cb67_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209aba457113c49475488f4cb7106b4cb67_del 2023-02-09 09:55:32,957 INFO [Listener at localhost/42557] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-09 09:55:32,957 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020995e5a9b60f5f459bb8b030b02f683434_del] 2023-02-09 09:55:32,957 DEBUG [Listener at localhost/42557] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:32,958 DEBUG [Listener at localhost/42557] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020995e5a9b60f5f459bb8b030b02f683434_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020995e5a9b60f5f459bb8b030b02f683434_del 2023-02-09 09:55:32,958 INFO [Listener at localhost/42557] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-09 09:55:32,962 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:32,965 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:32,967 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:32,968 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:32,973 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:32,973 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:32,974 INFO [Listener at localhost/42557] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-02-09 09:55:32,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-02-09 09:55:32,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-02-09 09:55:32,976 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:32,976 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:32,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 09:55:32,977 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936532977"}]},"ts":"1675936532977"} 2023-02-09 09:55:32,978 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-02-09 09:55:32,980 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-02-09 09:55:32,980 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, UNASSIGN}] 2023-02-09 09:55:32,982 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, UNASSIGN 2023-02-09 09:55:32,982 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, UNASSIGN 2023-02-09 09:55:32,982 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, UNASSIGN 2023-02-09 09:55:32,982 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=954cfb87e30cb0d4ca9c9e7b1988704f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:32,982 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=60d93642fa0ceecf3be9bd92ffd4e009, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:32,982 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=47fec7fd4b7f8dc5df9e778c5af07aa9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:32,983 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936532982"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936532982"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936532982"}]},"ts":"1675936532982"} 2023-02-09 09:55:32,983 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936532982"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936532982"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936532982"}]},"ts":"1675936532982"} 2023-02-09 09:55:32,983 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936532982"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936532982"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936532982"}]},"ts":"1675936532982"} 2023-02-09 09:55:32,984 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure 954cfb87e30cb0d4ca9c9e7b1988704f, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:32,984 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=91, state=RUNNABLE; CloseRegionProcedure 47fec7fd4b7f8dc5df9e778c5af07aa9, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:32,985 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure 60d93642fa0ceecf3be9bd92ffd4e009, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:33,077 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:33,077 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:33,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 09:55:33,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:33,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 954cfb87e30cb0d4ca9c9e7b1988704f, disabling compactions & flushes 2023-02-09 09:55:33,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:33,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:33,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. after waiting 0 ms 2023-02-09 09:55:33,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:33,140 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-09 09:55:33,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:33,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 954cfb87e30cb0d4ca9c9e7b1988704f: Waiting for close lock at 1675936533136Running coprocessor pre-close hooks at 1675936533136Disabling compacts and flushes for region at 1675936533136Disabling writes for close at 1675936533136Writing region close event to WAL at 1675936533137 (+1 ms)Running coprocessor post-close hooks at 1675936533141 (+4 ms)Closed at 1675936533141 2023-02-09 09:55:33,143 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:33,143 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:33,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 60d93642fa0ceecf3be9bd92ffd4e009, disabling compactions & flushes 2023-02-09 09:55:33,143 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:33,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:33,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. after waiting 0 ms 2023-02-09 09:55:33,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:33,144 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=954cfb87e30cb0d4ca9c9e7b1988704f, regionState=CLOSED 2023-02-09 09:55:33,144 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936533144"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936533144"}]},"ts":"1675936533144"} 2023-02-09 09:55:33,147 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-09 09:55:33,147 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure 954cfb87e30cb0d4ca9c9e7b1988704f, server=jenkins-hbase4.apache.org,42719,1675936416061 in 162 msec 2023-02-09 09:55:33,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-09 09:55:33,148 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, UNASSIGN in 167 msec 2023-02-09 09:55:33,149 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:33,149 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 60d93642fa0ceecf3be9bd92ffd4e009: Waiting for close lock at 1675936533143Running coprocessor pre-close hooks at 1675936533143Disabling compacts and flushes for region at 1675936533143Disabling writes for close at 1675936533143Writing region close event to WAL at 1675936533145 (+2 ms)Running coprocessor post-close hooks at 1675936533149 (+4 ms)Closed at 1675936533149 2023-02-09 09:55:33,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:33,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:33,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 47fec7fd4b7f8dc5df9e778c5af07aa9, disabling compactions & flushes 2023-02-09 09:55:33,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:33,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:33,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. after waiting 0 ms 2023-02-09 09:55:33,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:33,150 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=60d93642fa0ceecf3be9bd92ffd4e009, regionState=CLOSED 2023-02-09 09:55:33,151 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936533150"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936533150"}]},"ts":"1675936533150"} 2023-02-09 09:55:33,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=89 2023-02-09 09:55:33,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=89, state=SUCCESS; CloseRegionProcedure 60d93642fa0ceecf3be9bd92ffd4e009, server=jenkins-hbase4.apache.org,42719,1675936416061 in 167 msec 2023-02-09 09:55:33,154 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, UNASSIGN in 173 msec 2023-02-09 09:55:33,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-09 09:55:33,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:33,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: Waiting for close lock at 1675936533150Running coprocessor pre-close hooks at 1675936533150Disabling compacts and flushes for region at 1675936533150Disabling writes for close at 1675936533150Writing region close event to WAL at 1675936533151 (+1 ms)Running coprocessor post-close hooks at 1675936533155 (+4 ms)Closed at 1675936533155 2023-02-09 09:55:33,156 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:33,157 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=47fec7fd4b7f8dc5df9e778c5af07aa9, regionState=CLOSED 2023-02-09 09:55:33,157 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936533156"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936533156"}]},"ts":"1675936533156"} 2023-02-09 09:55:33,159 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=91 2023-02-09 09:55:33,159 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=91, state=SUCCESS; CloseRegionProcedure 47fec7fd4b7f8dc5df9e778c5af07aa9, server=jenkins-hbase4.apache.org,42719,1675936416061 in 174 msec 2023-02-09 09:55:33,160 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=88 2023-02-09 09:55:33,160 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, UNASSIGN in 179 msec 2023-02-09 09:55:33,161 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936533161"}]},"ts":"1675936533161"} 2023-02-09 09:55:33,162 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-02-09 09:55:33,163 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-02-09 09:55:33,164 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 189 msec 2023-02-09 09:55:33,279 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:33,279 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:33,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 09:55:33,280 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-02-09 09:55:33,281 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:33,281 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:33,290 INFO [Listener at localhost/42557] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1675936520620-1675936533290 2023-02-09 09:55:33,290 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:33,290 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:33,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 09:55:33,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1675936533291 (current time:1675936533291). 2023-02-09 09:55:33,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-09 09:55:33,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] zookeeper.ReadOnlyZKClient(139): Connect 0x6fd8184a to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:55:33,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1923aea1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:55:33,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:55:33,298 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37292, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:55:33,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6fd8184a to 127.0.0.1:64170 2023-02-09 09:55:33,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:55:33,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-09 09:55:33,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 09:55:33,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 09:55:33,302 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 09:55:33,302 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1675937133302 2023-02-09 09:55:33,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 09:55:33,302 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1675936520620-1675936533290 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-09 09:55:33,302 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:33,303 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-09 09:55:33,403 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 09:55:33,403 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:33,403 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:33,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 09:55:33,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 09:55:33,404 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-09 09:55:33,604 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 09:55:33,604 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:33,604 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:33,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 09:55:33,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 09:55:33,605 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-09 09:55:33,708 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-02-09 09:55:33,710 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 09:55:33,710 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 09:55:33,710 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 09:55:33,710 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 09:55:33,710 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 09:55:33,710 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 09:55:33,710 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 09:55:33,710 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 09:55:33,712 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/0eefc6df16b64194b2b7dfa903aca90e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/81e9ecae595346538da1437c3c4fe864_SeqId_29_] hfiles 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/0eefc6df16b64194b2b7dfa903aca90e 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/81e9ecae595346538da1437c3c4fe864_SeqId_29_ 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/0cebe03bad444a69912eafa88e28763c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/d0e4a898f1d44d9994a293fedf182eff_SeqId_29_] hfiles 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/0cebe03bad444a69912eafa88e28763c 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/d0e4a898f1d44d9994a293fedf182eff_SeqId_29_ 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/67390f8a971b4505b0c1d98ca7801392_SeqId_29_, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/6ce5b4c408e84375aa69ddcb6aece814] hfiles 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020909f30b5701ba4bccb7cd0c30d72be1fb, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209644d989a6911435aa486eb60cafc4702, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209989e11dde2e64d2984f01aa92f3931c9] hfiles 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/67390f8a971b4505b0c1d98ca7801392_SeqId_29_ 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020909f30b5701ba4bccb7cd0c30d72be1fb 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/6ce5b4c408e84375aa69ddcb6aece814 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209644d989a6911435aa486eb60cafc4702 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/c1f3fb59308a46df8f99c72d392adde3] hfiles 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209989e11dde2e64d2984f01aa92f3931c9 2023-02-09 09:55:33,713 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/c1f3fb59308a46df8f99c72d392adde3 2023-02-09 09:55:33,714 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/0897d8ff023c4c7ca70bab0eaf928a61] hfiles 2023-02-09 09:55:33,714 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/0897d8ff023c4c7ca70bab0eaf928a61 2023-02-09 09:55:33,714 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/a32261a786554d2191915906d58bb23b] hfiles 2023-02-09 09:55:33,714 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/a32261a786554d2191915906d58bb23b 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209373b70eeab504282afa2bb41ac264825, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302093fe0ef0f6c924387b73f84ab574e4d47, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209425f6e2534b24126a8a724fcf420bcf9_del, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302096ed54f417c6d4539b8912e8428a3387d, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302098e6acf6524e243f48bbaee4a9d740e3c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302092e33a279f6ac411a934a03f5f911ae76, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020953c0b8a3fc3a46f9bf1579619e7167b7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302095a0868e46faa47a5a907d4db4962f3ea, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302095d6070c7e4a04255b03e462902cab150_del, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209f2cf67b8d6e04a4a8852cb9a7cefca35, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020906ce2eb8cbc34ce3b7f7d5586c0714d8, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209196da5ec53964da7954a31b03f48e88c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209b1a803e2ec6747ed9d71da7de9db7d33, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209c6ef9ffacce24f6c80b5190072283490_del, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209ebbc78182e394898a21e1e6118635fba] hfiles 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209373b70eeab504282afa2bb41ac264825 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302093fe0ef0f6c924387b73f84ab574e4d47 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209425f6e2534b24126a8a724fcf420bcf9_del 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302096ed54f417c6d4539b8912e8428a3387d 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302098e6acf6524e243f48bbaee4a9d740e3c 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302092e33a279f6ac411a934a03f5f911ae76 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020953c0b8a3fc3a46f9bf1579619e7167b7 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302095a0868e46faa47a5a907d4db4962f3ea 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302095d6070c7e4a04255b03e462902cab150_del 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209f2cf67b8d6e04a4a8852cb9a7cefca35 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020906ce2eb8cbc34ce3b7f7d5586c0714d8 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209196da5ec53964da7954a31b03f48e88c 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209b1a803e2ec6747ed9d71da7de9db7d33 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209c6ef9ffacce24f6c80b5190072283490_del 2023-02-09 09:55:33,719 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209ebbc78182e394898a21e1e6118635fba 2023-02-09 09:55:33,906 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 09:55:33,906 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:33,906 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:33,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 09:55:33,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 09:55:33,907 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-09 09:55:34,122 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-02-09 09:55:34,122 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1675936520620-1675936533290 2023-02-09 09:55:34,122 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675936520620-1675936533290 2023-02-09 09:55:34,407 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 09:55:34,407 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:34,407 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:34,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 09:55:34,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 09:55:34,408 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-09 09:55:34,555 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675936520620-1675936533290 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/hbase-failsafe-snaptb-1675936520620-1675936533290 2023-02-09 09:55:35,369 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1675936520620-1675936533290 of table testCompactionWithHFileLink completed 2023-02-09 09:55:35,369 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675936520620-1675936533290 2023-02-09 09:55:35,370 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675936520620-1675936533290 2023-02-09 09:55:35,370 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1675936520620-1675936533290 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1675936533302Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1675936533708 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1675936520620-1675936533290 at 1675936534122 (+414 ms)Loading Region manifests for hbase-failsafe-snaptb-1675936520620-1675936533290 at 1675936534122Writing data manifest for hbase-failsafe-snaptb-1675936520620-1675936533290 at 1675936534125 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1675936520620-1675936533290 at 1675936534531 (+406 ms)Snapshot hbase-failsafe-snaptb-1675936520620-1675936533290 of table testCompactionWithHFileLink completed at 1675936535369 (+838 ms) 2023-02-09 09:55:35,371 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 09:55:35,372 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0700 sec 2023-02-09 09:55:35,408 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 09:55:35,408 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:35,408 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:35,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 09:55:35,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1675936520620-1675936533290 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-09 09:55:35,409 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:35,410 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:35,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1675936520620" table: "testCompactionWithHFileLink" creation_time: 1675936520630 type: FLUSH version: 2 ttl: 0 ) 2023-02-09 09:55:35,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1675936520620 as table=testCompactionWithHFileLink 2023-02-09 09:55:35,452 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:35,453 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:35,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 09:55:35,553 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:35,554 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:35,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 09:55:35,754 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:35,755 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:35,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 09:55:35,860 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-09 09:55:35,861 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-09 09:55:35,861 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-09 09:55:35,862 INFO [PEWorker-5] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 09:55:35,866 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1675936520620" table: "testCompactionWithHFileLink" creation_time: 1675936520630 type: FLUSH version: 2 ttl: 0 2023-02-09 09:55:35,866 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink 2023-02-09 09:55:35,871 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-02-09 09:55:35,874 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020909f30b5701ba4bccb7cd0c30d72be1fb to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020909f30b5701ba4bccb7cd0c30d72be1fb 2023-02-09 09:55:35,875 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209644d989a6911435aa486eb60cafc4702 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209644d989a6911435aa486eb60cafc4702 2023-02-09 09:55:35,876 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209989e11dde2e64d2984f01aa92f3931c9 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209989e11dde2e64d2984f01aa92f3931c9 2023-02-09 09:55:35,876 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023020960b2c1c05c9245d488e8aa1a05b0a7ff of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,880 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302099a8f83e7795d44e39f56953e4a247cad of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,883 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302099ccb9095ff214264a5e3aaf03df096ff of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,886 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230209a2bade1a5de94df69caf1593d3311949 of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,888 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302093860bd0a392948db8d92023da420e4c7 of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,891 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023020939be72032efb49678cbf53e19763c1d3 of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,894 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023020986b92434695045c8b7fd2993f6273d25 of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,896 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230209d3b01fffc2b84619a402d32d19edf15d of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,898 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302094e5d780559a742c0b6d327a4c8921b48 of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,901 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023020954c62db62733478eaaea1e942c7d0447 of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,903 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230209e52b294ef182415eb568aee0413d15f4 of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,906 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230209ea061cd2e689491fb9eaadae01f0b4e0 of snapshot=snaptb-1675936520620 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 09:55:35,911 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209425f6e2534b24126a8a724fcf420bcf9_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209425f6e2534b24126a8a724fcf420bcf9_del 2023-02-09 09:55:35,912 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209c6ef9ffacce24f6c80b5190072283490_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209c6ef9ffacce24f6c80b5190072283490_del 2023-02-09 09:55:35,913 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302095d6070c7e4a04255b03e462902cab150_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302095d6070c7e4a04255b03e462902cab150_del 2023-02-09 09:55:35,913 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:35,913 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:35,913 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:35,916 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/d0e4a898f1d44d9994a293fedf182eff_SeqId_29_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/d0e4a898f1d44d9994a293fedf182eff_SeqId_29_ 2023-02-09 09:55:35,916 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/81e9ecae595346538da1437c3c4fe864_SeqId_29_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/81e9ecae595346538da1437c3c4fe864_SeqId_29_ 2023-02-09 09:55:35,916 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/6ce5b4c408e84375aa69ddcb6aece814 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/6ce5b4c408e84375aa69ddcb6aece814 2023-02-09 09:55:35,918 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/0cebe03bad444a69912eafa88e28763c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/0cebe03bad444a69912eafa88e28763c 2023-02-09 09:55:35,918 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a6ca269ae6a84d519af27293a70fd223 of snapshot=snaptb-1675936520620 to region=954cfb87e30cb0d4ca9c9e7b1988704f table=testCompactionWithHFileLink 2023-02-09 09:55:35,918 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/67390f8a971b4505b0c1d98ca7801392_SeqId_29_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/67390f8a971b4505b0c1d98ca7801392_SeqId_29_ 2023-02-09 09:55:35,918 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/0eefc6df16b64194b2b7dfa903aca90e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/0eefc6df16b64194b2b7dfa903aca90e 2023-02-09 09:55:35,918 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e29bc24dc5194e0f848f9f3cb5264c45 of snapshot=snaptb-1675936520620 to region=47fec7fd4b7f8dc5df9e778c5af07aa9 table=testCompactionWithHFileLink 2023-02-09 09:55:35,918 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e0ed0a83b4f3420b92e9e0b5ff47afa1 of snapshot=snaptb-1675936520620 to region=60d93642fa0ceecf3be9bd92ffd4e009 table=testCompactionWithHFileLink 2023-02-09 09:55:35,927 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2b4c5e1a5d624abda847254ed690fee2 of snapshot=snaptb-1675936520620 to region=954cfb87e30cb0d4ca9c9e7b1988704f table=testCompactionWithHFileLink 2023-02-09 09:55:35,928 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0e819bfb780044c193e7afcbfd576d56 of snapshot=snaptb-1675936520620 to region=60d93642fa0ceecf3be9bd92ffd4e009 table=testCompactionWithHFileLink 2023-02-09 09:55:35,928 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cf96736150a04e7dafa94513af336383 of snapshot=snaptb-1675936520620 to region=47fec7fd4b7f8dc5df9e778c5af07aa9 table=testCompactionWithHFileLink 2023-02-09 09:55:35,936 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 76b31550dba244f68b84fc182e34921b of snapshot=snaptb-1675936520620 to region=47fec7fd4b7f8dc5df9e778c5af07aa9 table=testCompactionWithHFileLink 2023-02-09 09:55:35,936 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 593f9755d5f34a68b34e9e24cb4620c0 of snapshot=snaptb-1675936520620 to region=954cfb87e30cb0d4ca9c9e7b1988704f table=testCompactionWithHFileLink 2023-02-09 09:55:35,936 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 96179ee988fa437280604c295c8ed11c of snapshot=snaptb-1675936520620 to region=60d93642fa0ceecf3be9bd92ffd4e009 table=testCompactionWithHFileLink 2023-02-09 09:55:35,940 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ff516574d54647dd87b46ba263d32631 of snapshot=snaptb-1675936520620 to region=47fec7fd4b7f8dc5df9e778c5af07aa9 table=testCompactionWithHFileLink 2023-02-09 09:55:35,940 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4e9bb78e0c574fd186f7760fae94a250 of snapshot=snaptb-1675936520620 to region=954cfb87e30cb0d4ca9c9e7b1988704f table=testCompactionWithHFileLink 2023-02-09 09:55:35,940 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a93265fde28c461eaaa4e09ead22cce4 of snapshot=snaptb-1675936520620 to region=60d93642fa0ceecf3be9bd92ffd4e009 table=testCompactionWithHFileLink 2023-02-09 09:55:35,945 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/c1f3fb59308a46df8f99c72d392adde3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/c1f3fb59308a46df8f99c72d392adde3 2023-02-09 09:55:35,945 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 983958f5e14940aa8bae33560ee80c22 of snapshot=snaptb-1675936520620 to region=47fec7fd4b7f8dc5df9e778c5af07aa9 table=testCompactionWithHFileLink 2023-02-09 09:55:35,945 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/0897d8ff023c4c7ca70bab0eaf928a61 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/0897d8ff023c4c7ca70bab0eaf928a61 2023-02-09 09:55:35,945 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/a32261a786554d2191915906d58bb23b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/a32261a786554d2191915906d58bb23b 2023-02-09 09:55:35,945 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5ddaa897126749fc934e33a744c5d393 of snapshot=snaptb-1675936520620 to region=60d93642fa0ceecf3be9bd92ffd4e009 table=testCompactionWithHFileLink 2023-02-09 09:55:35,945 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink bbbc15242a304f499ce610c7b0bbc247 of snapshot=snaptb-1675936520620 to region=954cfb87e30cb0d4ca9c9e7b1988704f table=testCompactionWithHFileLink 2023-02-09 09:55:35,948 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f12746f2a255473696f11886211e7782 of snapshot=snaptb-1675936520620 to region=47fec7fd4b7f8dc5df9e778c5af07aa9 table=testCompactionWithHFileLink 2023-02-09 09:55:35,948 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ee5c35bfe19f4a0fa91bada511c58d14 of snapshot=snaptb-1675936520620 to region=954cfb87e30cb0d4ca9c9e7b1988704f table=testCompactionWithHFileLink 2023-02-09 09:55:35,949 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 06ce406e16c7488c8aeccab2d5d8b934 of snapshot=snaptb-1675936520620 to region=60d93642fa0ceecf3be9bd92ffd4e009 table=testCompactionWithHFileLink 2023-02-09 09:55:35,951 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 70bee5b32c144a35876afcc27e2e38b0 of snapshot=snaptb-1675936520620 to region=47fec7fd4b7f8dc5df9e778c5af07aa9 table=testCompactionWithHFileLink 2023-02-09 09:55:35,951 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1bb332c62cde4076a782e66eda402986 of snapshot=snaptb-1675936520620 to region=954cfb87e30cb0d4ca9c9e7b1988704f table=testCompactionWithHFileLink 2023-02-09 09:55:35,952 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9f82b2ac0f6948f293091af9dd3f0ddc of snapshot=snaptb-1675936520620 to region=60d93642fa0ceecf3be9bd92ffd4e009 table=testCompactionWithHFileLink 2023-02-09 09:55:35,956 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 838362fd9450402e84f7f7a8eddd6562 of snapshot=snaptb-1675936520620 to region=47fec7fd4b7f8dc5df9e778c5af07aa9 table=testCompactionWithHFileLink 2023-02-09 09:55:35,956 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 23e13d2980384fa29816e32999b316ec of snapshot=snaptb-1675936520620 to region=954cfb87e30cb0d4ca9c9e7b1988704f table=testCompactionWithHFileLink 2023-02-09 09:55:35,956 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9f08785f19784b10858a6bbf3398a180 of snapshot=snaptb-1675936520620 to region=60d93642fa0ceecf3be9bd92ffd4e009 table=testCompactionWithHFileLink 2023-02-09 09:55:35,962 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1675936520620" table: "testCompactionWithHFileLink" creation_time: 1675936520630 type: FLUSH version: 2 ttl: 0 2023-02-09 09:55:35,963 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675936535963"}]},"ts":"9223372036854775807"} 2023-02-09 09:55:35,963 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675936535963"}]},"ts":"9223372036854775807"} 2023-02-09 09:55:35,963 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675936535963"}]},"ts":"9223372036854775807"} 2023-02-09 09:55:35,965 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 09:55:35,966 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 47fec7fd4b7f8dc5df9e778c5af07aa9, NAME => 'testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 60d93642fa0ceecf3be9bd92ffd4e009, NAME => 'testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 954cfb87e30cb0d4ca9c9e7b1988704f, NAME => 'testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.', STARTKEY => '1', ENDKEY => '2'}] 2023-02-09 09:55:35,966 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936535964"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936535964"}]},"ts":"1675936535964"} 2023-02-09 09:55:35,966 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936535964"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936535964"}]},"ts":"1675936535964"} 2023-02-09 09:55:35,966 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936535964"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936535964"}]},"ts":"1675936535964"} 2023-02-09 09:55:35,967 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 09:55:35,968 INFO [PEWorker-5] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-02-09 09:55:35,968 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 47fec7fd4b7f8dc5df9e778c5af07aa9, NAME => 'testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 60d93642fa0ceecf3be9bd92ffd4e009, NAME => 'testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 954cfb87e30cb0d4ca9c9e7b1988704f, NAME => 'testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.', STARTKEY => '1', ENDKEY => '2'}] 2023-02-09 09:55:35,968 INFO [PEWorker-5] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1675936520620 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-02-09 09:55:35,971 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1675936520620" table: "testCompactionWithHFileLink" creation_time: 1675936520630 type: FLUSH version: 2 ttl: 0 ) in 530 msec 2023-02-09 09:55:36,056 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:36,056 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:36,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 09:55:36,057 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-02-09 09:55:36,057 INFO [Listener at localhost/42557] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1675936520620-1675936533290 2023-02-09 09:55:36,057 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:36,057 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:36,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1675936520620-1675936533290" 2023-02-09 09:55:36,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1675936520620-1675936533290 2023-02-09 09:55:36,065 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:36,065 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:36,065 INFO [Listener at localhost/42557] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-02-09 09:55:36,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$10(2372): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-02-09 09:55:36,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-02-09 09:55:36,077 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:36,077 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:36,077 INFO [PEWorker-4] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-02-09 09:55:36,077 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936536077"}]},"ts":"1675936536077"} 2023-02-09 09:55:36,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 09:55:36,078 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-09 09:55:36,081 DEBUG [PEWorker-4] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-02-09 09:55:36,081 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, ASSIGN}] 2023-02-09 09:55:36,082 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, ASSIGN 2023-02-09 09:55:36,082 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, ASSIGN 2023-02-09 09:55:36,082 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, ASSIGN 2023-02-09 09:55:36,083 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-09 09:55:36,083 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-09 09:55:36,083 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-09 09:55:36,178 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:36,178 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:36,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 09:55:36,234 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=47fec7fd4b7f8dc5df9e778c5af07aa9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:36,234 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=954cfb87e30cb0d4ca9c9e7b1988704f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:36,234 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936536234"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936536234"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936536234"}]},"ts":"1675936536234"} 2023-02-09 09:55:36,234 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=60d93642fa0ceecf3be9bd92ffd4e009, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:36,234 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936536234"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936536234"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936536234"}]},"ts":"1675936536234"} 2023-02-09 09:55:36,234 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936536234"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936536234"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936536234"}]},"ts":"1675936536234"} 2023-02-09 09:55:36,236 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure 47fec7fd4b7f8dc5df9e778c5af07aa9, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:36,236 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=99, state=RUNNABLE; OpenRegionProcedure 954cfb87e30cb0d4ca9c9e7b1988704f, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:36,237 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure 60d93642fa0ceecf3be9bd92ffd4e009, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:36,379 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:36,379 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:36,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 09:55:36,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:36,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 47fec7fd4b7f8dc5df9e778c5af07aa9, NAME => 'testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 09:55:36,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:36,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:36,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:36,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:36,392 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:36,393 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1 2023-02-09 09:55:36,393 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1 2023-02-09 09:55:36,393 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47fec7fd4b7f8dc5df9e778c5af07aa9 columnFamilyName family1 2023-02-09 09:55:36,410 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-76b31550dba244f68b84fc182e34921b 2023-02-09 09:55:36,416 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-cf96736150a04e7dafa94513af336383 2023-02-09 09:55:36,421 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-e29bc24dc5194e0f848f9f3cb5264c45 2023-02-09 09:55:36,427 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-ff516574d54647dd87b46ba263d32631 2023-02-09 09:55:36,427 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(310): Store=47fec7fd4b7f8dc5df9e778c5af07aa9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:36,427 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:36,428 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2 2023-02-09 09:55:36,428 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2 2023-02-09 09:55:36,429 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47fec7fd4b7f8dc5df9e778c5af07aa9 columnFamilyName family2 2023-02-09 09:55:36,438 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-70bee5b32c144a35876afcc27e2e38b0 2023-02-09 09:55:36,449 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-838362fd9450402e84f7f7a8eddd6562 2023-02-09 09:55:36,454 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-983958f5e14940aa8bae33560ee80c22 2023-02-09 09:55:36,459 DEBUG [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-f12746f2a255473696f11886211e7782 2023-02-09 09:55:36,459 INFO [StoreOpener-47fec7fd4b7f8dc5df9e778c5af07aa9-1] regionserver.HStore(310): Store=47fec7fd4b7f8dc5df9e778c5af07aa9/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:36,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:36,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:36,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:55:36,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:55:36,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 47fec7fd4b7f8dc5df9e778c5af07aa9; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66475611, jitterRate=-0.00943620502948761}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:55:36,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: Running coprocessor pre-open hook at 1675936536391Writing region info on filesystem at 1675936536391Initializing all the Stores at 1675936536391Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936536392 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936536392Cleaning up temporary data from old regions at 1675936536462 (+70 ms)Cleaning up detritus from prior splits at 1675936536462Running coprocessor post-open hooks at 1675936536465 (+3 ms)Region opened successfully at 1675936536465 2023-02-09 09:55:36,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9., pid=101, masterSystemTime=1675936536387 2023-02-09 09:55:36,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:55:36,466 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 09:55:36,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 09:55:36,466 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 09:55:36,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:36,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:36,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:36,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 60d93642fa0ceecf3be9bd92ffd4e009, NAME => 'testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 09:55:36,468 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=47fec7fd4b7f8dc5df9e778c5af07aa9, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:36,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:36,468 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936536468"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936536468"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936536468"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936536468"}]},"ts":"1675936536468"} 2023-02-09 09:55:36,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:36,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:36,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:36,470 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 09:55:36,470 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1912): 47fec7fd4b7f8dc5df9e778c5af07aa9/family2 is initiating minor compaction (all files) 2023-02-09 09:55:36,470 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 47fec7fd4b7f8dc5df9e778c5af07aa9/family2 in testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:36,470 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 09:55:36,470 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:36,470 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1912): 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 is initiating minor compaction (all files) 2023-02-09 09:55:36,470 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-983958f5e14940aa8bae33560ee80c22, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-f12746f2a255473696f11886211e7782, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-70bee5b32c144a35876afcc27e2e38b0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-838362fd9450402e84f7f7a8eddd6562] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp, totalSize=20.4 K 2023-02-09 09:55:36,470 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 in testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:55:36,470 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-e29bc24dc5194e0f848f9f3cb5264c45, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-cf96736150a04e7dafa94513af336383, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-76b31550dba244f68b84fc182e34921b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-ff516574d54647dd87b46ba263d32631] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp, totalSize=21.6 K 2023-02-09 09:55:36,471 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-983958f5e14940aa8bae33560ee80c22, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513467 2023-02-09 09:55:36,471 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-e29bc24dc5194e0f848f9f3cb5264c45, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513467 2023-02-09 09:55:36,472 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-f12746f2a255473696f11886211e7782, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:36,472 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1 2023-02-09 09:55:36,472 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1 2023-02-09 09:55:36,472 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-02-09 09:55:36,472 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure 47fec7fd4b7f8dc5df9e778c5af07aa9, server=jenkins-hbase4.apache.org,42719,1675936416061 in 234 msec 2023-02-09 09:55:36,472 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60d93642fa0ceecf3be9bd92ffd4e009 columnFamilyName family1 2023-02-09 09:55:36,472 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-cf96736150a04e7dafa94513af336383, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:36,473 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-70bee5b32c144a35876afcc27e2e38b0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517024 2023-02-09 09:55:36,473 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-76b31550dba244f68b84fc182e34921b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517024 2023-02-09 09:55:36,473 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=47fec7fd4b7f8dc5df9e778c5af07aa9, ASSIGN in 391 msec 2023-02-09 09:55:36,474 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-838362fd9450402e84f7f7a8eddd6562, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518825 2023-02-09 09:55:36,474 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-ff516574d54647dd87b46ba263d32631, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518825 2023-02-09 09:55:36,488 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-0e819bfb780044c193e7afcbfd576d56 2023-02-09 09:55:36,497 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-96179ee988fa437280604c295c8ed11c 2023-02-09 09:55:36,507 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-a93265fde28c461eaaa4e09ead22cce4 2023-02-09 09:55:36,511 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 47fec7fd4b7f8dc5df9e778c5af07aa9#family1#compaction#113 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:36,511 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] throttle.PressureAwareThroughputController(145): 47fec7fd4b7f8dc5df9e778c5af07aa9#family2#compaction#114 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:36,519 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-e0ed0a83b4f3420b92e9e0b5ff47afa1 2023-02-09 09:55:36,519 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(310): Store=60d93642fa0ceecf3be9bd92ffd4e009/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:36,520 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:36,521 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2 2023-02-09 09:55:36,521 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2 2023-02-09 09:55:36,521 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60d93642fa0ceecf3be9bd92ffd4e009 columnFamilyName family2 2023-02-09 09:55:36,529 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-06ce406e16c7488c8aeccab2d5d8b934 2023-02-09 09:55:36,542 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family1/2ca71894e4744245a4f5a63cb69f89df as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/2ca71894e4744245a4f5a63cb69f89df 2023-02-09 09:55:36,542 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-5ddaa897126749fc934e33a744c5d393 2023-02-09 09:55:36,546 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 of 47fec7fd4b7f8dc5df9e778c5af07aa9 into 2ca71894e4744245a4f5a63cb69f89df(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:55:36,546 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:36,546 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f08785f19784b10858a6bbf3398a180 2023-02-09 09:55:36,546 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9., storeName=47fec7fd4b7f8dc5df9e778c5af07aa9/family1, priority=12, startTime=1675936536466; duration=0sec 2023-02-09 09:55:36,547 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:55:36,551 DEBUG [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f82b2ac0f6948f293091af9dd3f0ddc 2023-02-09 09:55:36,551 INFO [StoreOpener-60d93642fa0ceecf3be9bd92ffd4e009-1] regionserver.HStore(310): Store=60d93642fa0ceecf3be9bd92ffd4e009/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:36,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:36,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:36,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:55:36,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:55:36,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 60d93642fa0ceecf3be9bd92ffd4e009; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72758958, jitterRate=0.08419296145439148}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:55:36,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 60d93642fa0ceecf3be9bd92ffd4e009: Running coprocessor pre-open hook at 1675936536469Writing region info on filesystem at 1675936536469Initializing all the Stores at 1675936536469Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936536470 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936536470Cleaning up temporary data from old regions at 1675936536553 (+83 ms)Cleaning up detritus from prior splits at 1675936536553Running coprocessor post-open hooks at 1675936536555 (+2 ms)Region opened successfully at 1675936536555 2023-02-09 09:55:36,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009., pid=103, masterSystemTime=1675936536387 2023-02-09 09:55:36,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:55:36,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 09:55:36,557 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 09:55:36,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:36,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:36,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:36,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 954cfb87e30cb0d4ca9c9e7b1988704f, NAME => 'testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 09:55:36,558 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=60d93642fa0ceecf3be9bd92ffd4e009, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:36,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:36,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:36,558 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675936536558"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936536558"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936536558"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936536558"}]},"ts":"1675936536558"} 2023-02-09 09:55:36,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:36,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:36,559 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 09:55:36,559 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1912): 60d93642fa0ceecf3be9bd92ffd4e009/family1 is initiating minor compaction (all files) 2023-02-09 09:55:36,559 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 60d93642fa0ceecf3be9bd92ffd4e009/family1 in testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:36,559 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-e0ed0a83b4f3420b92e9e0b5ff47afa1, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-0e819bfb780044c193e7afcbfd576d56, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-96179ee988fa437280604c295c8ed11c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-a93265fde28c461eaaa4e09ead22cce4] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp, totalSize=21.6 K 2023-02-09 09:55:36,559 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:36,559 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-e0ed0a83b4f3420b92e9e0b5ff47afa1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513465 2023-02-09 09:55:36,560 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-0e819bfb780044c193e7afcbfd576d56, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:36,560 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1 2023-02-09 09:55:36,560 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1 2023-02-09 09:55:36,561 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 954cfb87e30cb0d4ca9c9e7b1988704f columnFamilyName family1 2023-02-09 09:55:36,561 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-96179ee988fa437280604c295c8ed11c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517023 2023-02-09 09:55:36,561 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=98 2023-02-09 09:55:36,561 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure 60d93642fa0ceecf3be9bd92ffd4e009, server=jenkins-hbase4.apache.org,42719,1675936416061 in 322 msec 2023-02-09 09:55:36,562 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-a93265fde28c461eaaa4e09ead22cce4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518824 2023-02-09 09:55:36,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60d93642fa0ceecf3be9bd92ffd4e009, ASSIGN in 480 msec 2023-02-09 09:55:36,567 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-2b4c5e1a5d624abda847254ed690fee2 2023-02-09 09:55:36,573 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-4e9bb78e0c574fd186f7760fae94a250 2023-02-09 09:55:36,579 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-e29bc24dc5194e0f848f9f3cb5264c45, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-cf96736150a04e7dafa94513af336383, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-76b31550dba244f68b84fc182e34921b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-ff516574d54647dd87b46ba263d32631] to archive 2023-02-09 09:55:36,579 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:36,581 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-593f9755d5f34a68b34e9e24cb4620c0 2023-02-09 09:55:36,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-e29bc24dc5194e0f848f9f3cb5264c45 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-e29bc24dc5194e0f848f9f3cb5264c45 2023-02-09 09:55:36,582 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 60d93642fa0ceecf3be9bd92ffd4e009#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:36,583 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-cf96736150a04e7dafa94513af336383 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-cf96736150a04e7dafa94513af336383 2023-02-09 09:55:36,584 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-76b31550dba244f68b84fc182e34921b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-76b31550dba244f68b84fc182e34921b 2023-02-09 09:55:36,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-ff516574d54647dd87b46ba263d32631 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-ff516574d54647dd87b46ba263d32631 2023-02-09 09:55:36,586 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-a6ca269ae6a84d519af27293a70fd223 2023-02-09 09:55:36,588 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(310): Store=954cfb87e30cb0d4ca9c9e7b1988704f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:36,588 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:36,589 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2 2023-02-09 09:55:36,589 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2 2023-02-09 09:55:36,589 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 954cfb87e30cb0d4ca9c9e7b1988704f columnFamilyName family2 2023-02-09 09:55:36,595 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-1bb332c62cde4076a782e66eda402986 2023-02-09 09:55:36,599 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-23e13d2980384fa29816e32999b316ec 2023-02-09 09:55:36,603 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-bbbc15242a304f499ce610c7b0bbc247 2023-02-09 09:55:36,608 DEBUG [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(539): loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-ee5c35bfe19f4a0fa91bada511c58d14 2023-02-09 09:55:36,609 INFO [StoreOpener-954cfb87e30cb0d4ca9c9e7b1988704f-1] regionserver.HStore(310): Store=954cfb87e30cb0d4ca9c9e7b1988704f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:36,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:36,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:36,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:55:36,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:55:36,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 954cfb87e30cb0d4ca9c9e7b1988704f; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68468537, jitterRate=0.020260706543922424}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:55:36,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 954cfb87e30cb0d4ca9c9e7b1988704f: Running coprocessor pre-open hook at 1675936536558Writing region info on filesystem at 1675936536558Initializing all the Stores at 1675936536559 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936536559Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936536559Cleaning up temporary data from old regions at 1675936536611 (+52 ms)Cleaning up detritus from prior splits at 1675936536611Running coprocessor post-open hooks at 1675936536613 (+2 ms)Region opened successfully at 1675936536613 2023-02-09 09:55:36,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f., pid=102, masterSystemTime=1675936536387 2023-02-09 09:55:36,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 09:55:36,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-09 09:55:36,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:36,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:36,615 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=954cfb87e30cb0d4ca9c9e7b1988704f, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:36,616 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936536615"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936536615"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936536615"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936536615"}]},"ts":"1675936536615"} 2023-02-09 09:55:36,618 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=99 2023-02-09 09:55:36,618 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=99, state=SUCCESS; OpenRegionProcedure 954cfb87e30cb0d4ca9c9e7b1988704f, server=jenkins-hbase4.apache.org,42719,1675936416061 in 381 msec 2023-02-09 09:55:36,619 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-02-09 09:55:36,619 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=954cfb87e30cb0d4ca9c9e7b1988704f, ASSIGN in 537 msec 2023-02-09 09:55:36,621 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936536621"}]},"ts":"1675936536621"} 2023-02-09 09:55:36,622 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-09 09:55:36,624 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-02-09 09:55:36,626 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 555 msec 2023-02-09 09:55:36,681 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:36,681 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:36,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 09:55:36,682 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-02-09 09:55:36,695 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:36,703 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:36,708 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:36,709 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:36,715 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 09:55:36,727 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-09 09:55:36,727 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:36,727 DEBUG [Listener at localhost/42557] zookeeper.ReadOnlyZKClient(139): Connect 0x7e586a95 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:55:36,732 DEBUG [Listener at localhost/42557] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9c55fc7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:55:36,732 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:36,732 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:36,732 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:36,944 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/.tmp/family2/47880787fe98413aa3c604f9476e8a03 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/47880787fe98413aa3c604f9476e8a03 2023-02-09 09:55:36,949 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 47fec7fd4b7f8dc5df9e778c5af07aa9/family2 of 47fec7fd4b7f8dc5df9e778c5af07aa9 into 47880787fe98413aa3c604f9476e8a03(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:55:36,949 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:36,949 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9., storeName=47fec7fd4b7f8dc5df9e778c5af07aa9/family2, priority=12, startTime=1675936536466; duration=0sec 2023-02-09 09:55:36,949 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-09 09:55:36,949 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 09:55:36,951 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 09:55:36,951 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1912): 60d93642fa0ceecf3be9bd92ffd4e009/family2 is initiating minor compaction (all files) 2023-02-09 09:55:36,951 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 60d93642fa0ceecf3be9bd92ffd4e009/family2 in testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:55:36,951 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-5ddaa897126749fc934e33a744c5d393, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-06ce406e16c7488c8aeccab2d5d8b934, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f82b2ac0f6948f293091af9dd3f0ddc, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f08785f19784b10858a6bbf3398a180] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp, totalSize=20.4 K 2023-02-09 09:55:36,952 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-5ddaa897126749fc934e33a744c5d393, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513465 2023-02-09 09:55:36,953 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-06ce406e16c7488c8aeccab2d5d8b934, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:36,953 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f82b2ac0f6948f293091af9dd3f0ddc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517023 2023-02-09 09:55:36,954 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f08785f19784b10858a6bbf3398a180, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518824 2023-02-09 09:55:36,969 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] throttle.PressureAwareThroughputController(145): 60d93642fa0ceecf3be9bd92ffd4e009#family2#compaction#116 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:36,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-983958f5e14940aa8bae33560ee80c22, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-f12746f2a255473696f11886211e7782, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-70bee5b32c144a35876afcc27e2e38b0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-838362fd9450402e84f7f7a8eddd6562] to archive 2023-02-09 09:55:36,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:36,977 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-983958f5e14940aa8bae33560ee80c22 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-983958f5e14940aa8bae33560ee80c22 2023-02-09 09:55:36,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-f12746f2a255473696f11886211e7782 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-f12746f2a255473696f11886211e7782 2023-02-09 09:55:36,979 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-70bee5b32c144a35876afcc27e2e38b0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-70bee5b32c144a35876afcc27e2e38b0 2023-02-09 09:55:36,980 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-838362fd9450402e84f7f7a8eddd6562 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family2/testCompactionWithHFileLink=47fec7fd4b7f8dc5df9e778c5af07aa9-838362fd9450402e84f7f7a8eddd6562 2023-02-09 09:55:37,627 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302098d8e56d968a14d8eb2081c85710a3874 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302098d8e56d968a14d8eb2081c85710a3874 2023-02-09 09:55:37,627 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302092ca97f7972e84f1faefe7d0c90939d2b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302092ca97f7972e84f1faefe7d0c90939d2b 2023-02-09 09:55:37,627 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209a2554a821e0c466ab8de0054b2283e18 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209a2554a821e0c466ab8de0054b2283e18 2023-02-09 09:55:37,628 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:37,628 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:37,628 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:37,644 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:55:37,647 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54442, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:55:37,655 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,655 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,655 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,655 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:55:37,664 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42506, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:55:37,665 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,665 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,665 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:37,666 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,666 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:37,696 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/a11f0379fc114aa49c3d3ab30e19cd8b first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-02-09 09:55:37,696 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/94b3142f6b8a4fb6a5e57f94514ca561 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-02-09 09:55:37,697 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/8833790c866940eda40a71b990b815da first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-02-09 09:55:37,697 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/94b3142f6b8a4fb6a5e57f94514ca561}] 2023-02-09 09:55:37,697 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/a11f0379fc114aa49c3d3ab30e19cd8b}] 2023-02-09 09:55:37,697 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=32 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/8833790c866940eda40a71b990b815da}] 2023-02-09 09:55:37,704 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/94b3142f6b8a4fb6a5e57f94514ca561 for inclusion in 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 2023-02-09 09:55:37,704 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/8833790c866940eda40a71b990b815da for inclusion in 60d93642fa0ceecf3be9bd92ffd4e009/family1 2023-02-09 09:55:37,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-02-09 09:55:37,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-02-09 09:55:37,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 09:55:37,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 09:55:37,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:37,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: 2023-02-09 09:55:37,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/8833790c866940eda40a71b990b815da to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__ra3uctales6a0hmm6qra91ncaml8i7dgnd8gbe1pl7otk2hurq20i1i3dcro1kg/family1/8833790c866940eda40a71b990b815da 2023-02-09 09:55:37,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/94b3142f6b8a4fb6a5e57f94514ca561 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__7i65kl6lunb0iq1dv21tvknr0uq8um4172d56925elpd7jd1imdfpif6ve8jbnm3/family1/94b3142f6b8a4fb6a5e57f94514ca561 2023-02-09 09:55:37,708 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/a11f0379fc114aa49c3d3ab30e19cd8b for inclusion in 954cfb87e30cb0d4ca9c9e7b1988704f/family1 2023-02-09 09:55:37,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-02-09 09:55:37,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 09:55:37,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__ra3uctales6a0hmm6qra91ncaml8i7dgnd8gbe1pl7otk2hurq20i1i3dcro1kg/family1/8833790c866940eda40a71b990b815da as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/7ade61f159bb4a3483dccae3a4c83cce_SeqId_34_ 2023-02-09 09:55:37,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:37,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__ra3uctales6a0hmm6qra91ncaml8i7dgnd8gbe1pl7otk2hurq20i1i3dcro1kg/family1/8833790c866940eda40a71b990b815da into 60d93642fa0ceecf3be9bd92ffd4e009/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/7ade61f159bb4a3483dccae3a4c83cce_SeqId_34_ - updating store file list. 2023-02-09 09:55:37,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__7i65kl6lunb0iq1dv21tvknr0uq8um4172d56925elpd7jd1imdfpif6ve8jbnm3/family1/94b3142f6b8a4fb6a5e57f94514ca561 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/26a9d37d7e7b4eb183d93ce93c32f4bc_SeqId_36_ 2023-02-09 09:55:37,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/a11f0379fc114aa49c3d3ab30e19cd8b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__427bfv938oktpf95c685asggg80km4sq125u5nmn87e6ivc8knjv6fsrbqbkeak9/family1/a11f0379fc114aa49c3d3ab30e19cd8b 2023-02-09 09:55:37,713 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__7i65kl6lunb0iq1dv21tvknr0uq8um4172d56925elpd7jd1imdfpif6ve8jbnm3/family1/94b3142f6b8a4fb6a5e57f94514ca561 into 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/26a9d37d7e7b4eb183d93ce93c32f4bc_SeqId_36_ - updating store file list. 2023-02-09 09:55:37,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__427bfv938oktpf95c685asggg80km4sq125u5nmn87e6ivc8knjv6fsrbqbkeak9/family1/a11f0379fc114aa49c3d3ab30e19cd8b as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/809c26cdb45a4e03af26b1d93d03cb78_SeqId_34_ 2023-02-09 09:55:37,715 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__427bfv938oktpf95c685asggg80km4sq125u5nmn87e6ivc8knjv6fsrbqbkeak9/family1/a11f0379fc114aa49c3d3ab30e19cd8b into 954cfb87e30cb0d4ca9c9e7b1988704f/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/809c26cdb45a4e03af26b1d93d03cb78_SeqId_34_ - updating store file list. 2023-02-09 09:55:37,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/7ade61f159bb4a3483dccae3a4c83cce_SeqId_34_ into 60d93642fa0ceecf3be9bd92ffd4e009/family1 2023-02-09 09:55:37,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__ra3uctales6a0hmm6qra91ncaml8i7dgnd8gbe1pl7otk2hurq20i1i3dcro1kg/family1/8833790c866940eda40a71b990b815da into 60d93642fa0ceecf3be9bd92ffd4e009/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/7ade61f159bb4a3483dccae3a4c83cce_SeqId_34_) 2023-02-09 09:55:37,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__ra3uctales6a0hmm6qra91ncaml8i7dgnd8gbe1pl7otk2hurq20i1i3dcro1kg/family1/8833790c866940eda40a71b990b815da 2023-02-09 09:55:37,717 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/26a9d37d7e7b4eb183d93ce93c32f4bc_SeqId_36_ into 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 2023-02-09 09:55:37,717 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__7i65kl6lunb0iq1dv21tvknr0uq8um4172d56925elpd7jd1imdfpif6ve8jbnm3/family1/94b3142f6b8a4fb6a5e57f94514ca561 into 47fec7fd4b7f8dc5df9e778c5af07aa9/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/family1/26a9d37d7e7b4eb183d93ce93c32f4bc_SeqId_36_) 2023-02-09 09:55:37,718 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:37,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__7i65kl6lunb0iq1dv21tvknr0uq8um4172d56925elpd7jd1imdfpif6ve8jbnm3/family1/94b3142f6b8a4fb6a5e57f94514ca561 2023-02-09 09:55:37,719 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:37,719 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/809c26cdb45a4e03af26b1d93d03cb78_SeqId_34_ into 954cfb87e30cb0d4ca9c9e7b1988704f/family1 2023-02-09 09:55:37,719 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__427bfv938oktpf95c685asggg80km4sq125u5nmn87e6ivc8knjv6fsrbqbkeak9/family1/a11f0379fc114aa49c3d3ab30e19cd8b into 954cfb87e30cb0d4ca9c9e7b1988704f/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/809c26cdb45a4e03af26b1d93d03cb78_SeqId_34_) 2023-02-09 09:55:37,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testCompactionWithHFileLink__427bfv938oktpf95c685asggg80km4sq125u5nmn87e6ivc8knjv6fsrbqbkeak9/family1/a11f0379fc114aa49c3d3ab30e19cd8b 2023-02-09 09:55:37,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:37,720 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302094e5d780559a742c0b6d327a4c8921b48 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302094e5d780559a742c0b6d327a4c8921b48 2023-02-09 09:55:37,721 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:37,721 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023020954c62db62733478eaaea1e942c7d0447 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023020954c62db62733478eaaea1e942c7d0447 2023-02-09 09:55:37,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-09 09:55:37,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(1912): 954cfb87e30cb0d4ca9c9e7b1988704f/family1 is initiating minor compaction (all files) 2023-02-09 09:55:37,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1e19b3b5; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-09 09:55:37,722 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:37,723 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209e52b294ef182415eb568aee0413d15f4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209e52b294ef182415eb568aee0413d15f4 2023-02-09 09:55:37,723 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:37,724 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302093860bd0a392948db8d92023da420e4c7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302093860bd0a392948db8d92023da420e4c7 2023-02-09 09:55:37,724 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209ea061cd2e689491fb9eaadae01f0b4e0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209ea061cd2e689491fb9eaadae01f0b4e0 2023-02-09 09:55:37,724 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 09:55:37,724 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 09:55:37,724 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:37,724 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:37,725 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023020939be72032efb49678cbf53e19763c1d3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023020939be72032efb49678cbf53e19763c1d3 2023-02-09 09:55:37,726 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023020960b2c1c05c9245d488e8aa1a05b0a7ff to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023020960b2c1c05c9245d488e8aa1a05b0a7ff 2023-02-09 09:55:37,726 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023020986b92434695045c8b7fd2993f6273d25 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023020986b92434695045c8b7fd2993f6273d25 2023-02-09 09:55:37,728 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302099a8f83e7795d44e39f56953e4a247cad to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302099a8f83e7795d44e39f56953e4a247cad 2023-02-09 09:55:37,728 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209d3b01fffc2b84619a402d32d19edf15d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209d3b01fffc2b84619a402d32d19edf15d 2023-02-09 09:55:37,728 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 09:55:37,728 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 09:55:37,729 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:37,729 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302099ccb9095ff214264a5e3aaf03df096ff to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302099ccb9095ff214264a5e3aaf03df096ff 2023-02-09 09:55:37,730 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230209a2bade1a5de94df69caf1593d3311949 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230209a2bade1a5de94df69caf1593d3311949 2023-02-09 09:55:37,730 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 09:55:37,730 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 09:55:37,731 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 09:55:37,731 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 09:55:37,734 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:37,736 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:37,743 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:37,744 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:37,751 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:37,752 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:37,773 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=280 (was 245) - Thread LEAK? -, OpenFileDescriptor=746 (was 610) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=111 (was 169), ProcessCount=168 (was 168), AvailableMemoryMB=3241 (was 3353) 2023-02-09 09:55:37,785 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=280, OpenFileDescriptor=746, MaxFileDescriptor=60000, SystemLoadAverage=111, ProcessCount=168, AvailableMemoryMB=3240 2023-02-09 09:55:37,786 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,786 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:37,790 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$15(2997): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-02-09 09:55:37,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-02-09 09:55:37,794 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,794 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:37,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-09 09:55:37,796 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 09:55:37,798 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 7 msec 2023-02-09 09:55:37,803 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family1/1303494ddc964b77b4c63ad30b6dbe15 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/1303494ddc964b77b4c63ad30b6dbe15 2023-02-09 09:55:37,807 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 60d93642fa0ceecf3be9bd92ffd4e009/family1 of 60d93642fa0ceecf3be9bd92ffd4e009 into 1303494ddc964b77b4c63ad30b6dbe15(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:37,807 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:37,807 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009., storeName=60d93642fa0ceecf3be9bd92ffd4e009/family1, priority=12, startTime=1675936536556; duration=1sec 2023-02-09 09:55:37,808 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-09 09:55:37,808 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 954cfb87e30cb0d4ca9c9e7b1988704f/family1 in testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:37,808 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-a6ca269ae6a84d519af27293a70fd223, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-2b4c5e1a5d624abda847254ed690fee2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-593f9755d5f34a68b34e9e24cb4620c0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-4e9bb78e0c574fd186f7760fae94a250, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/809c26cdb45a4e03af26b1d93d03cb78_SeqId_34_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp, totalSize=28.9 K 2023-02-09 09:55:37,808 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-a6ca269ae6a84d519af27293a70fd223, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513467 2023-02-09 09:55:37,809 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-2b4c5e1a5d624abda847254ed690fee2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:37,809 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-593f9755d5f34a68b34e9e24cb4620c0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517024 2023-02-09 09:55:37,810 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-4e9bb78e0c574fd186f7760fae94a250, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518824 2023-02-09 09:55:37,810 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 809c26cdb45a4e03af26b1d93d03cb78_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1675936513467 2023-02-09 09:55:37,823 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 954cfb87e30cb0d4ca9c9e7b1988704f#family1#compaction#117 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-09 09:55:37,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-e0ed0a83b4f3420b92e9e0b5ff47afa1, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-0e819bfb780044c193e7afcbfd576d56, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-96179ee988fa437280604c295c8ed11c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-a93265fde28c461eaaa4e09ead22cce4] to archive 2023-02-09 09:55:37,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:37,878 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-e0ed0a83b4f3420b92e9e0b5ff47afa1 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-e0ed0a83b4f3420b92e9e0b5ff47afa1 2023-02-09 09:55:37,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-0e819bfb780044c193e7afcbfd576d56 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-0e819bfb780044c193e7afcbfd576d56 2023-02-09 09:55:37,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-96179ee988fa437280604c295c8ed11c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-96179ee988fa437280604c295c8ed11c 2023-02-09 09:55:37,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-a93265fde28c461eaaa4e09ead22cce4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family1/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-a93265fde28c461eaaa4e09ead22cce4 2023-02-09 09:55:37,895 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,895 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:37,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-09 09:55:37,896 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,896 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:37,898 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 09:55:37,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-02-09 09:55:37,900 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:55:37,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-02-09 09:55:37,901 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:37,901 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:37,901 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:55:37,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 09:55:37,903 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:55:37,903 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569 2023-02-09 09:55:37,903 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374 2023-02-09 09:55:37,903 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569 empty. 2023-02-09 09:55:37,903 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078 empty. 2023-02-09 09:55:37,903 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374 empty. 2023-02-09 09:55:37,904 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569 2023-02-09 09:55:37,904 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:55:37,904 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374 2023-02-09 09:55:37,904 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-02-09 09:55:38,002 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:38,002 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:38,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 09:55:38,188 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/.tmp/family2/86ff4969897a465fbec7b694a0b93828 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/86ff4969897a465fbec7b694a0b93828 2023-02-09 09:55:38,191 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 60d93642fa0ceecf3be9bd92ffd4e009/family2 of 60d93642fa0ceecf3be9bd92ffd4e009 into 86ff4969897a465fbec7b694a0b93828(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:38,192 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 60d93642fa0ceecf3be9bd92ffd4e009: 2023-02-09 09:55:38,192 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009., storeName=60d93642fa0ceecf3be9bd92ffd4e009/family2, priority=12, startTime=1675936536557; duration=1sec 2023-02-09 09:55:38,192 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 09:55:38,192 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 09:55:38,193 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 09:55:38,193 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1912): 954cfb87e30cb0d4ca9c9e7b1988704f/family2 is initiating minor compaction (all files) 2023-02-09 09:55:38,194 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 954cfb87e30cb0d4ca9c9e7b1988704f/family2 in testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:55:38,194 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-bbbc15242a304f499ce610c7b0bbc247, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-ee5c35bfe19f4a0fa91bada511c58d14, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-1bb332c62cde4076a782e66eda402986, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-23e13d2980384fa29816e32999b316ec] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp, totalSize=20.4 K 2023-02-09 09:55:38,194 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-bbbc15242a304f499ce610c7b0bbc247, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936513467 2023-02-09 09:55:38,195 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-ee5c35bfe19f4a0fa91bada511c58d14, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936515246 2023-02-09 09:55:38,195 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-1bb332c62cde4076a782e66eda402986, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936517024 2023-02-09 09:55:38,195 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-23e13d2980384fa29816e32999b316ec, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936518824 2023-02-09 09:55:38,203 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:38,203 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:38,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 09:55:38,206 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] throttle.PressureAwareThroughputController(145): 954cfb87e30cb0d4ca9c9e7b1988704f#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-09 09:55:38,275 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-5ddaa897126749fc934e33a744c5d393, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-06ce406e16c7488c8aeccab2d5d8b934, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f82b2ac0f6948f293091af9dd3f0ddc, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f08785f19784b10858a6bbf3398a180] to archive 2023-02-09 09:55:38,275 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:38,277 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-5ddaa897126749fc934e33a744c5d393 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-5ddaa897126749fc934e33a744c5d393 2023-02-09 09:55:38,278 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-06ce406e16c7488c8aeccab2d5d8b934 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-06ce406e16c7488c8aeccab2d5d8b934 2023-02-09 09:55:38,279 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f82b2ac0f6948f293091af9dd3f0ddc to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f82b2ac0f6948f293091af9dd3f0ddc 2023-02-09 09:55:38,280 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f08785f19784b10858a6bbf3398a180 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/family2/testCompactionWithHFileLink=60d93642fa0ceecf3be9bd92ffd4e009-9f08785f19784b10858a6bbf3398a180 2023-02-09 09:55:38,311 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-02-09 09:55:38,312 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => d1ea6fc17b9a3bc112723fe71be3a078, NAME => 'ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:55:38,313 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7671): creating {ENCODED => e761366629ce85046adc8c312fb15374, NAME => 'ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:55:38,313 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7671): creating {ENCODED => e20e3543303de552ba42845d95488569, NAME => 'ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:55:38,325 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:38,325 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1603): Closing d1ea6fc17b9a3bc112723fe71be3a078, disabling compactions & flushes 2023-02-09 09:55:38,325 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:38,325 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:38,325 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. after waiting 0 ms 2023-02-09 09:55:38,325 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:38,325 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:38,326 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1557): Region close journal for d1ea6fc17b9a3bc112723fe71be3a078: Waiting for close lock at 1675936538325Disabling compacts and flushes for region at 1675936538325Disabling writes for close at 1675936538325Writing region close event to WAL at 1675936538325Closed at 1675936538325 2023-02-09 09:55:38,504 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:38,504 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:38,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 09:55:38,686 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:55:38,722 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:38,722 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1603): Closing e761366629ce85046adc8c312fb15374, disabling compactions & flushes 2023-02-09 09:55:38,722 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:38,722 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:38,722 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. after waiting 0 ms 2023-02-09 09:55:38,722 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:38,722 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:38,722 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1557): Region close journal for e761366629ce85046adc8c312fb15374: Waiting for close lock at 1675936538722Disabling compacts and flushes for region at 1675936538722Disabling writes for close at 1675936538722Writing region close event to WAL at 1675936538722Closed at 1675936538722 2023-02-09 09:55:38,726 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:38,726 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1603): Closing e20e3543303de552ba42845d95488569, disabling compactions & flushes 2023-02-09 09:55:38,726 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:38,726 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:38,726 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. after waiting 0 ms 2023-02-09 09:55:38,726 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:38,726 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:38,726 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1557): Region close journal for e20e3543303de552ba42845d95488569: Waiting for close lock at 1675936538726Disabling compacts and flushes for region at 1675936538726Disabling writes for close at 1675936538726Writing region close event to WAL at 1675936538726Closed at 1675936538726 2023-02-09 09:55:38,728 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:55:38,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675936538729"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936538729"}]},"ts":"1675936538729"} 2023-02-09 09:55:38,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675936538729"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936538729"}]},"ts":"1675936538729"} 2023-02-09 09:55:38,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675936538729"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936538729"}]},"ts":"1675936538729"} 2023-02-09 09:55:38,730 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 09:55:38,731 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:55:38,731 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936538731"}]},"ts":"1675936538731"} 2023-02-09 09:55:38,732 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-02-09 09:55:38,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d1ea6fc17b9a3bc112723fe71be3a078, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e20e3543303de552ba42845d95488569, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e761366629ce85046adc8c312fb15374, ASSIGN}] 2023-02-09 09:55:38,736 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d1ea6fc17b9a3bc112723fe71be3a078, ASSIGN 2023-02-09 09:55:38,736 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e761366629ce85046adc8c312fb15374, ASSIGN 2023-02-09 09:55:38,736 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e20e3543303de552ba42845d95488569, ASSIGN 2023-02-09 09:55:38,737 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d1ea6fc17b9a3bc112723fe71be3a078, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:55:38,737 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e20e3543303de552ba42845d95488569, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:55:38,737 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e761366629ce85046adc8c312fb15374, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:55:38,890 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=e20e3543303de552ba42845d95488569, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:38,890 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=e761366629ce85046adc8c312fb15374, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:38,890 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d1ea6fc17b9a3bc112723fe71be3a078, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:38,890 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675936538890"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936538890"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936538890"}]},"ts":"1675936538890"} 2023-02-09 09:55:38,890 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675936538890"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936538890"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936538890"}]},"ts":"1675936538890"} 2023-02-09 09:55:38,890 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675936538890"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936538890"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936538890"}]},"ts":"1675936538890"} 2023-02-09 09:55:38,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure e20e3543303de552ba42845d95488569, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:38,892 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure d1ea6fc17b9a3bc112723fe71be3a078, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:38,894 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure e761366629ce85046adc8c312fb15374, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:39,005 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:39,005 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:39,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 09:55:39,038 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family1/c129d8edbc8a46e48f9885488f8a639c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/c129d8edbc8a46e48f9885488f8a639c 2023-02-09 09:55:39,043 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 954cfb87e30cb0d4ca9c9e7b1988704f/family1 of 954cfb87e30cb0d4ca9c9e7b1988704f into c129d8edbc8a46e48f9885488f8a639c(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:39,043 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:39,043 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1675936537722; duration=1sec 2023-02-09 09:55:39,043 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 09:55:39,044 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 09:55:39,044 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-09 09:55:39,044 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-09 09:55:39,044 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. because compaction request was cancelled 2023-02-09 09:55:39,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:39,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e761366629ce85046adc8c312fb15374, NAME => 'ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 09:55:39,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction e761366629ce85046adc8c312fb15374 2023-02-09 09:55:39,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:39,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e761366629ce85046adc8c312fb15374 2023-02-09 09:55:39,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e761366629ce85046adc8c312fb15374 2023-02-09 09:55:39,047 INFO [StoreOpener-e761366629ce85046adc8c312fb15374-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e761366629ce85046adc8c312fb15374 2023-02-09 09:55:39,048 DEBUG [StoreOpener-e761366629ce85046adc8c312fb15374-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1 2023-02-09 09:55:39,048 DEBUG [StoreOpener-e761366629ce85046adc8c312fb15374-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1 2023-02-09 09:55:39,049 INFO [StoreOpener-e761366629ce85046adc8c312fb15374-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e761366629ce85046adc8c312fb15374 columnFamilyName family1 2023-02-09 09:55:39,050 INFO [StoreOpener-e761366629ce85046adc8c312fb15374-1] regionserver.HStore(310): Store=e761366629ce85046adc8c312fb15374/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:39,050 INFO [StoreOpener-e761366629ce85046adc8c312fb15374-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e761366629ce85046adc8c312fb15374 2023-02-09 09:55:39,051 DEBUG [StoreOpener-e761366629ce85046adc8c312fb15374-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2 2023-02-09 09:55:39,051 DEBUG [StoreOpener-e761366629ce85046adc8c312fb15374-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2 2023-02-09 09:55:39,052 INFO [StoreOpener-e761366629ce85046adc8c312fb15374-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e761366629ce85046adc8c312fb15374 columnFamilyName family2 2023-02-09 09:55:39,053 INFO [StoreOpener-e761366629ce85046adc8c312fb15374-1] regionserver.HStore(310): Store=e761366629ce85046adc8c312fb15374/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:39,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374 2023-02-09 09:55:39,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374 2023-02-09 09:55:39,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:55:39,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e761366629ce85046adc8c312fb15374 2023-02-09 09:55:39,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:55:39,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e761366629ce85046adc8c312fb15374; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59412565, jitterRate=-0.11468379199504852}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:55:39,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e761366629ce85046adc8c312fb15374: Running coprocessor pre-open hook at 1675936539046Writing region info on filesystem at 1675936539046Initializing all the Stores at 1675936539046Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936539046Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936539047 (+1 ms)Cleaning up temporary data from old regions at 1675936539055 (+8 ms)Cleaning up detritus from prior splits at 1675936539055Running coprocessor post-open hooks at 1675936539059 (+4 ms)Region opened successfully at 1675936539059 2023-02-09 09:55:39,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374., pid=111, masterSystemTime=1675936539043 2023-02-09 09:55:39,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:39,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:39,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:39,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e20e3543303de552ba42845d95488569, NAME => 'ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 09:55:39,062 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=e761366629ce85046adc8c312fb15374, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:39,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction e20e3543303de552ba42845d95488569 2023-02-09 09:55:39,062 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675936539062"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936539062"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936539062"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936539062"}]},"ts":"1675936539062"} 2023-02-09 09:55:39,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:39,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e20e3543303de552ba42845d95488569 2023-02-09 09:55:39,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e20e3543303de552ba42845d95488569 2023-02-09 09:55:39,063 INFO [StoreOpener-e20e3543303de552ba42845d95488569-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e20e3543303de552ba42845d95488569 2023-02-09 09:55:39,065 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=108 2023-02-09 09:55:39,065 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure e761366629ce85046adc8c312fb15374, server=jenkins-hbase4.apache.org,42719,1675936416061 in 169 msec 2023-02-09 09:55:39,065 DEBUG [StoreOpener-e20e3543303de552ba42845d95488569-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1 2023-02-09 09:55:39,065 DEBUG [StoreOpener-e20e3543303de552ba42845d95488569-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1 2023-02-09 09:55:39,066 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e761366629ce85046adc8c312fb15374, ASSIGN in 330 msec 2023-02-09 09:55:39,066 INFO [StoreOpener-e20e3543303de552ba42845d95488569-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e20e3543303de552ba42845d95488569 columnFamilyName family1 2023-02-09 09:55:39,067 INFO [StoreOpener-e20e3543303de552ba42845d95488569-1] regionserver.HStore(310): Store=e20e3543303de552ba42845d95488569/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:39,067 INFO [StoreOpener-e20e3543303de552ba42845d95488569-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e20e3543303de552ba42845d95488569 2023-02-09 09:55:39,068 DEBUG [StoreOpener-e20e3543303de552ba42845d95488569-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2 2023-02-09 09:55:39,068 DEBUG [StoreOpener-e20e3543303de552ba42845d95488569-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2 2023-02-09 09:55:39,068 INFO [StoreOpener-e20e3543303de552ba42845d95488569-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e20e3543303de552ba42845d95488569 columnFamilyName family2 2023-02-09 09:55:39,069 INFO [StoreOpener-e20e3543303de552ba42845d95488569-1] regionserver.HStore(310): Store=e20e3543303de552ba42845d95488569/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:39,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569 2023-02-09 09:55:39,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569 2023-02-09 09:55:39,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:55:39,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e20e3543303de552ba42845d95488569 2023-02-09 09:55:39,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:55:39,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-a6ca269ae6a84d519af27293a70fd223, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-2b4c5e1a5d624abda847254ed690fee2, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-593f9755d5f34a68b34e9e24cb4620c0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-4e9bb78e0c574fd186f7760fae94a250, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/809c26cdb45a4e03af26b1d93d03cb78_SeqId_34_] to archive 2023-02-09 09:55:39,076 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:39,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e20e3543303de552ba42845d95488569; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73021930, jitterRate=0.08811154961585999}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:55:39,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e20e3543303de552ba42845d95488569: Running coprocessor pre-open hook at 1675936539062Writing region info on filesystem at 1675936539062Initializing all the Stores at 1675936539063 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936539063Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936539063Cleaning up temporary data from old regions at 1675936539071 (+8 ms)Cleaning up detritus from prior splits at 1675936539071Running coprocessor post-open hooks at 1675936539076 (+5 ms)Region opened successfully at 1675936539076 2023-02-09 09:55:39,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569., pid=109, masterSystemTime=1675936539043 2023-02-09 09:55:39,077 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-a6ca269ae6a84d519af27293a70fd223 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-a6ca269ae6a84d519af27293a70fd223 2023-02-09 09:55:39,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:39,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:39,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:39,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d1ea6fc17b9a3bc112723fe71be3a078, NAME => 'ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 09:55:39,078 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=e20e3543303de552ba42845d95488569, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:39,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:55:39,078 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675936539078"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936539078"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936539078"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936539078"}]},"ts":"1675936539078"} 2023-02-09 09:55:39,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:39,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:55:39,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:55:39,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-2b4c5e1a5d624abda847254ed690fee2 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-2b4c5e1a5d624abda847254ed690fee2 2023-02-09 09:55:39,080 INFO [StoreOpener-d1ea6fc17b9a3bc112723fe71be3a078-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:55:39,080 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-593f9755d5f34a68b34e9e24cb4620c0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-593f9755d5f34a68b34e9e24cb4620c0 2023-02-09 09:55:39,081 DEBUG [StoreOpener-d1ea6fc17b9a3bc112723fe71be3a078-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1 2023-02-09 09:55:39,081 DEBUG [StoreOpener-d1ea6fc17b9a3bc112723fe71be3a078-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1 2023-02-09 09:55:39,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=107 2023-02-09 09:55:39,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure e20e3543303de552ba42845d95488569, server=jenkins-hbase4.apache.org,42719,1675936416061 in 188 msec 2023-02-09 09:55:39,081 INFO [StoreOpener-d1ea6fc17b9a3bc112723fe71be3a078-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d1ea6fc17b9a3bc112723fe71be3a078 columnFamilyName family1 2023-02-09 09:55:39,082 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-4e9bb78e0c574fd186f7760fae94a250 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-4e9bb78e0c574fd186f7760fae94a250 2023-02-09 09:55:39,082 INFO [StoreOpener-d1ea6fc17b9a3bc112723fe71be3a078-1] regionserver.HStore(310): Store=d1ea6fc17b9a3bc112723fe71be3a078/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:39,082 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=e20e3543303de552ba42845d95488569, ASSIGN in 346 msec 2023-02-09 09:55:39,082 INFO [StoreOpener-d1ea6fc17b9a3bc112723fe71be3a078-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:55:39,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/809c26cdb45a4e03af26b1d93d03cb78_SeqId_34_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family1/809c26cdb45a4e03af26b1d93d03cb78_SeqId_34_ 2023-02-09 09:55:39,084 DEBUG [StoreOpener-d1ea6fc17b9a3bc112723fe71be3a078-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2 2023-02-09 09:55:39,084 DEBUG [StoreOpener-d1ea6fc17b9a3bc112723fe71be3a078-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2 2023-02-09 09:55:39,084 INFO [StoreOpener-d1ea6fc17b9a3bc112723fe71be3a078-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d1ea6fc17b9a3bc112723fe71be3a078 columnFamilyName family2 2023-02-09 09:55:39,085 INFO [StoreOpener-d1ea6fc17b9a3bc112723fe71be3a078-1] regionserver.HStore(310): Store=d1ea6fc17b9a3bc112723fe71be3a078/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:39,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:55:39,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:55:39,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:55:39,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:55:39,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:55:39,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d1ea6fc17b9a3bc112723fe71be3a078; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62768405, jitterRate=-0.0646778792142868}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:55:39,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d1ea6fc17b9a3bc112723fe71be3a078: Running coprocessor pre-open hook at 1675936539079Writing region info on filesystem at 1675936539079Initializing all the Stores at 1675936539079Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936539079Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936539079Cleaning up temporary data from old regions at 1675936539086 (+7 ms)Cleaning up detritus from prior splits at 1675936539086Running coprocessor post-open hooks at 1675936539089 (+3 ms)Region opened successfully at 1675936539089 2023-02-09 09:55:39,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078., pid=110, masterSystemTime=1675936539043 2023-02-09 09:55:39,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:39,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:39,091 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d1ea6fc17b9a3bc112723fe71be3a078, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:39,091 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675936539091"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936539091"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936539091"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936539091"}]},"ts":"1675936539091"} 2023-02-09 09:55:39,093 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=106 2023-02-09 09:55:39,093 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure d1ea6fc17b9a3bc112723fe71be3a078, server=jenkins-hbase4.apache.org,42719,1675936416061 in 200 msec 2023-02-09 09:55:39,094 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-09 09:55:39,094 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d1ea6fc17b9a3bc112723fe71be3a078, ASSIGN in 358 msec 2023-02-09 09:55:39,095 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:55:39,095 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936539095"}]},"ts":"1675936539095"} 2023-02-09 09:55:39,096 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-02-09 09:55:39,097 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:55:39,098 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2000 sec 2023-02-09 09:55:39,424 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/.tmp/family2/6d5f1faa3dce49f1b6e818403b1e5402 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/6d5f1faa3dce49f1b6e818403b1e5402 2023-02-09 09:55:39,428 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 954cfb87e30cb0d4ca9c9e7b1988704f/family2 of 954cfb87e30cb0d4ca9c9e7b1988704f into 6d5f1faa3dce49f1b6e818403b1e5402(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:39,428 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 954cfb87e30cb0d4ca9c9e7b1988704f: 2023-02-09 09:55:39,428 INFO [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f., storeName=954cfb87e30cb0d4ca9c9e7b1988704f/family2, priority=12, startTime=1675936536614; duration=1sec 2023-02-09 09:55:39,428 DEBUG [RS:0;jenkins-hbase4:42719-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:55:39,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-bbbc15242a304f499ce610c7b0bbc247, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-ee5c35bfe19f4a0fa91bada511c58d14, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-1bb332c62cde4076a782e66eda402986, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-23e13d2980384fa29816e32999b316ec] to archive 2023-02-09 09:55:39,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:39,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-bbbc15242a304f499ce610c7b0bbc247 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-bbbc15242a304f499ce610c7b0bbc247 2023-02-09 09:55:39,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-ee5c35bfe19f4a0fa91bada511c58d14 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-ee5c35bfe19f4a0fa91bada511c58d14 2023-02-09 09:55:39,479 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-1bb332c62cde4076a782e66eda402986 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-1bb332c62cde4076a782e66eda402986 2023-02-09 09:55:39,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-23e13d2980384fa29816e32999b316ec to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/family2/testCompactionWithHFileLink=954cfb87e30cb0d4ca9c9e7b1988704f-23e13d2980384fa29816e32999b316ec 2023-02-09 09:55:40,006 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:40,006 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:40,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 09:55:40,007 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-02-09 09:55:40,013 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:40,013 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:40,013 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:40,015 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:40,015 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:40,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:40,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 09:55:40,016 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 09:55:40,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:40,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 09:55:40,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 09:55:40,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:40,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:40,018 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:40,018 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:40,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:40,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:40,019 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:40,019 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:40,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:40,019 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:40,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:40,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:40,019 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 09:55:40,019 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 09:55:40,019 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 09:55:40,019 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:40,020 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 09:55:40,020 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:40,020 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:40,020 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:55:40,020 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:40,020 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. started... 2023-02-09 09:55:40,020 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:40,020 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. started... 2023-02-09 09:55:40,020 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. started... 2023-02-09 09:55:40,022 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing e761366629ce85046adc8c312fb15374 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 09:55:40,022 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing d1ea6fc17b9a3bc112723fe71be3a078 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 09:55:40,022 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e20e3543303de552ba42845d95488569 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 09:55:40,034 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020982a8bf3cea684043908fe83939a92f7e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023020982a8bf3cea684043908fe83939a92f7e 2023-02-09 09:55:40,435 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209ab8b43afc1234d91ac391e703e7b9efc to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209ab8b43afc1234d91ac391e703e7b9efc 2023-02-09 09:55:40,435 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209e4bb4a937a4c404aa00e3a00509f2423 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209e4bb4a937a4c404aa00e3a00509f2423 2023-02-09 09:55:40,441 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/889da0fcb06a487c9f1db7052bc204f0 2023-02-09 09:55:40,840 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/00b95c158ea64237b8b5beb648ab7e6c 2023-02-09 09:55:40,840 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/69ef322672764861914b723a2974a804 2023-02-09 09:55:40,853 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302090c611c7cc5054100bf145be64adf2052 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302090c611c7cc5054100bf145be64adf2052 2023-02-09 09:55:41,253 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209c6957d82a1874a25bfbce589fc7a24dd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230209c6957d82a1874a25bfbce589fc7a24dd 2023-02-09 09:55:41,253 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020984795c1ef18b461283bbd45ad0d0015c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023020984795c1ef18b461283bbd45ad0d0015c 2023-02-09 09:55:41,257 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/ad7ea0bfc5274b198b80d02828cf2979 2023-02-09 09:55:41,262 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/889da0fcb06a487c9f1db7052bc204f0 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/889da0fcb06a487c9f1db7052bc204f0 2023-02-09 09:55:41,264 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/57d2f1a0571a43cc972b61678a5de71a 2023-02-09 09:55:41,265 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/c6ece1a28565414a8a281770ba47ae85 2023-02-09 09:55:41,267 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/889da0fcb06a487c9f1db7052bc204f0, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 09:55:41,268 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/ad7ea0bfc5274b198b80d02828cf2979 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/ad7ea0bfc5274b198b80d02828cf2979 2023-02-09 09:55:41,272 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/69ef322672764861914b723a2974a804 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/69ef322672764861914b723a2974a804 2023-02-09 09:55:41,273 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/00b95c158ea64237b8b5beb648ab7e6c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/00b95c158ea64237b8b5beb648ab7e6c 2023-02-09 09:55:41,275 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/ad7ea0bfc5274b198b80d02828cf2979, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 09:55:41,276 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/69ef322672764861914b723a2974a804, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 09:55:41,276 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d1ea6fc17b9a3bc112723fe71be3a078 in 1254ms, sequenceid=5, compaction requested=false 2023-02-09 09:55:41,276 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-02-09 09:55:41,277 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for d1ea6fc17b9a3bc112723fe71be3a078: 2023-02-09 09:55:41,277 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:41,277 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:55:41,277 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/57d2f1a0571a43cc972b61678a5de71a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/57d2f1a0571a43cc972b61678a5de71a 2023-02-09 09:55:41,278 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/00b95c158ea64237b8b5beb648ab7e6c, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 09:55:41,279 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/c6ece1a28565414a8a281770ba47ae85 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/c6ece1a28565414a8a281770ba47ae85 2023-02-09 09:55:41,281 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/57d2f1a0571a43cc972b61678a5de71a, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 09:55:41,283 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for e761366629ce85046adc8c312fb15374 in 1261ms, sequenceid=5, compaction requested=false 2023-02-09 09:55:41,284 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for e761366629ce85046adc8c312fb15374: 2023-02-09 09:55:41,284 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:41,284 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/c6ece1a28565414a8a281770ba47ae85, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 09:55:41,284 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:55:41,284 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for e20e3543303de552ba42845d95488569 in 1262ms, sequenceid=5, compaction requested=false 2023-02-09 09:55:41,284 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e20e3543303de552ba42845d95488569: 2023-02-09 09:55:41,284 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:41,284 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:55:41,284 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:55:41,285 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,285 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 09:55:41,285 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 09:55:41,286 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,286 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:41,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:41,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:41,286 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:41,286 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:41,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:41,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:41,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:41,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:41,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 09:55:41,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f0a0678[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:41,288 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 09:55:41,288 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:41,290 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:41,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:41,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:41,291 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 09:55:41,291 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 09:55:41,291 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:41,291 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,291 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:41,292 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 09:55:41,292 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:41,292 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,292 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 09:55:41,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:41,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:41,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:41,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:41,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:41,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:41,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:41,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:41,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 09:55:41,295 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 09:55:41,295 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:41,295 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:41,295 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 09:55:41,295 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:41,297 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:41,297 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:41,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:41,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:41,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:41,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:41,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:41,297 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:41,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:41,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:41,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:41,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:41,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:41,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:41,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:41,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:41,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:41,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,308 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,309 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,310 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,311 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:41,311 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:41,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:41,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 09:55:41,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:41,311 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:41,311 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:41,311 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:41,311 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:41,311 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:41,311 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:41,311 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:41,311 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:41,311 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:41,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:41,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:41,411 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:41,411 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:41,412 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:41,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:41,413 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:41,414 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:41,414 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:41,417 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:41,417 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:41,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:41,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:41,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 09:55:41,421 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 09:55:41,421 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:41,421 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 09:55:41,421 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 09:55:41,421 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:41,422 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:41,424 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,427 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:41,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:41,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:41,428 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:41,428 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:41,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:41,428 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:41,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:41,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:41,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 09:55:41,429 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 09:55:41,429 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 09:55:41,429 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 09:55:41,429 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:41,429 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 09:55:41,429 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:41,429 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. started... 2023-02-09 09:55:41,429 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:41,430 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. started... 2023-02-09 09:55:41,430 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e20e3543303de552ba42845d95488569 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 09:55:41,430 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing d1ea6fc17b9a3bc112723fe71be3a078 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 09:55:41,430 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:55:41,430 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:41,430 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:41,430 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. started... 2023-02-09 09:55:41,430 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing e761366629ce85046adc8c312fb15374 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 09:55:41,462 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209393c9ddff1404b6787c731db3601f641 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209393c9ddff1404b6787c731db3601f641 2023-02-09 09:55:41,853 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302093452bcf255c8414b9797804e72631572 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302093452bcf255c8414b9797804e72631572 2023-02-09 09:55:41,857 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209dd62bbbf56d74e85aadf1d1dba1174c3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209dd62bbbf56d74e85aadf1d1dba1174c3 2023-02-09 09:55:41,863 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/e355d4d2ac65452884fb91b4ceabb476 2023-02-09 09:55:41,869 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/c9f92d88613043afae05fcb787bc3dc0 2023-02-09 09:55:41,883 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302098afc4479d8574da49b3fb945b68f2865 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302098afc4479d8574da49b3fb945b68f2865 2023-02-09 09:55:42,259 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/13450e5d1e014992b4869db2e225ad98 2023-02-09 09:55:42,276 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302098ff9224950ad4ec59555e4f3459d4f26 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302098ff9224950ad4ec59555e4f3459d4f26 2023-02-09 09:55:42,288 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/4c5c80582f734dfa81b3b137b3e57d25 2023-02-09 09:55:42,291 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/c9f92d88613043afae05fcb787bc3dc0 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/c9f92d88613043afae05fcb787bc3dc0 2023-02-09 09:55:42,294 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/c9f92d88613043afae05fcb787bc3dc0, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 09:55:42,294 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/4c5c80582f734dfa81b3b137b3e57d25 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4c5c80582f734dfa81b3b137b3e57d25 2023-02-09 09:55:42,297 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4c5c80582f734dfa81b3b137b3e57d25, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 09:55:42,298 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for e20e3543303de552ba42845d95488569 in 868ms, sequenceid=9, compaction requested=false 2023-02-09 09:55:42,298 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e20e3543303de552ba42845d95488569: 2023-02-09 09:55:42,298 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:42,298 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:55:42,672 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209ba05e2c265df479cb4c18c464da91028 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230209ba05e2c265df479cb4c18c464da91028 2023-02-09 09:55:42,681 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/2a7a08fe30004da58fa5a39f16f03163 2023-02-09 09:55:42,685 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/e355d4d2ac65452884fb91b4ceabb476 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/e355d4d2ac65452884fb91b4ceabb476 2023-02-09 09:55:42,688 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/e355d4d2ac65452884fb91b4ceabb476, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 09:55:42,689 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/2a7a08fe30004da58fa5a39f16f03163 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/2a7a08fe30004da58fa5a39f16f03163 2023-02-09 09:55:42,692 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/2a7a08fe30004da58fa5a39f16f03163, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 09:55:42,693 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for e761366629ce85046adc8c312fb15374 in 1263ms, sequenceid=9, compaction requested=false 2023-02-09 09:55:42,693 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for e761366629ce85046adc8c312fb15374: 2023-02-09 09:55:42,693 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:42,693 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:55:43,077 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/aedd485feb1944d2b3da52218930a973 2023-02-09 09:55:43,081 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/13450e5d1e014992b4869db2e225ad98 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/13450e5d1e014992b4869db2e225ad98 2023-02-09 09:55:43,084 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/13450e5d1e014992b4869db2e225ad98, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 09:55:43,084 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/aedd485feb1944d2b3da52218930a973 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/aedd485feb1944d2b3da52218930a973 2023-02-09 09:55:43,087 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/aedd485feb1944d2b3da52218930a973, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 09:55:43,088 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d1ea6fc17b9a3bc112723fe71be3a078 in 1658ms, sequenceid=9, compaction requested=false 2023-02-09 09:55:43,088 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for d1ea6fc17b9a3bc112723fe71be3a078: 2023-02-09 09:55:43,088 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:43,088 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:55:43,089 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:55:43,089 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,089 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 09:55:43,089 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 09:55:43,090 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,090 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:43,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:43,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:43,090 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:43,090 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:43,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:43,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:43,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:43,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:43,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 09:55:43,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@679470d2[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:43,092 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 09:55:43,092 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:43,093 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:43,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:43,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:43,093 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 09:55:43,093 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 09:55:43,093 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:43,093 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,093 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:43,103 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 09:55:43,103 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,103 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:43,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:43,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:43,103 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 09:55:43,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:43,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:43,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:43,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:43,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:43,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:43,105 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 09:55:43,105 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 09:55:43,105 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:43,105 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:43,105 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 09:55:43,105 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:43,106 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:43,107 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:43,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:43,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:43,107 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:43,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:43,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:43,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:43,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:43,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:43,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:43,107 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:43,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:43,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:43,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:43,108 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:43,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:43,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:43,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:43,110 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:43,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:43,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:43,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 09:55:43,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:43,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:43,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:43,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:43,111 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:43,111 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:43,111 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:43,111 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:43,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:43,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:43,211 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:43,211 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:43,211 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:43,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:43,213 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:43,213 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:43,213 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:43,215 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:43,215 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:43,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:43,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:43,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 09:55:43,217 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 09:55:43,217 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:43,217 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 09:55:43,217 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 09:55:43,217 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:43,217 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:43,218 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,219 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:43,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:43,219 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:43,219 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:43,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:43,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 09:55:43,220 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 09:55:43,220 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 09:55:43,220 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 09:55:43,220 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:43,220 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 09:55:43,220 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:43,220 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. started... 2023-02-09 09:55:43,220 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:55:43,220 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:43,220 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:43,220 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:43,220 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. started... 2023-02-09 09:55:43,220 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e20e3543303de552ba42845d95488569 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 09:55:43,220 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. started... 2023-02-09 09:55:43,220 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing e761366629ce85046adc8c312fb15374 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 09:55:43,221 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing d1ea6fc17b9a3bc112723fe71be3a078 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 09:55:43,233 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209966fd99175c8491889009329b2ac5b1e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230209966fd99175c8491889009329b2ac5b1e 2023-02-09 09:55:43,634 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209eaaa82aebcd649e9ba7d35c4b332f533 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209eaaa82aebcd649e9ba7d35c4b332f533 2023-02-09 09:55:43,637 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302090816e9849cb84adfbbdff21c88c71452 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302090816e9849cb84adfbbdff21c88c71452 2023-02-09 09:55:43,637 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/ddf9c4bd83bb47269b0338fb177eb98e 2023-02-09 09:55:44,048 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/deb9e58575f7460182e3eb47e6781301 2023-02-09 09:55:44,062 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/ab3af65f8bcd48258c9ecfd09ac620c1 2023-02-09 09:55:44,085 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302097b2436b1893444c78d178c1b11bd91a3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302097b2436b1893444c78d178c1b11bd91a3 2023-02-09 09:55:44,464 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209f2d06770472f4e6bb1d17dc89a703944 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230209f2d06770472f4e6bb1d17dc89a703944 2023-02-09 09:55:44,475 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302090f209de95fa742f5874a39edb92244ca to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302090f209de95fa742f5874a39edb92244ca 2023-02-09 09:55:44,491 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/f4c90badd95f40b2812cceb7aebe0547 2023-02-09 09:55:44,494 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/ddf9c4bd83bb47269b0338fb177eb98e as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/ddf9c4bd83bb47269b0338fb177eb98e 2023-02-09 09:55:44,497 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/ddf9c4bd83bb47269b0338fb177eb98e, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 09:55:44,498 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/f4c90badd95f40b2812cceb7aebe0547 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/f4c90badd95f40b2812cceb7aebe0547 2023-02-09 09:55:44,500 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/f4c90badd95f40b2812cceb7aebe0547, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 09:55:44,501 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d1ea6fc17b9a3bc112723fe71be3a078 in 1281ms, sequenceid=13, compaction requested=true 2023-02-09 09:55:44,501 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for d1ea6fc17b9a3bc112723fe71be3a078: 2023-02-09 09:55:44,501 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:44,501 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:55:44,547 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:55:44,868 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/4a99bfe656a04c79910bbdd354966cff 2023-02-09 09:55:44,872 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/deb9e58575f7460182e3eb47e6781301 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/deb9e58575f7460182e3eb47e6781301 2023-02-09 09:55:44,876 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/deb9e58575f7460182e3eb47e6781301, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 09:55:44,877 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/4a99bfe656a04c79910bbdd354966cff as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4a99bfe656a04c79910bbdd354966cff 2023-02-09 09:55:44,880 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/5fad7861847f410886fe32271ae155dd 2023-02-09 09:55:44,880 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4a99bfe656a04c79910bbdd354966cff, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 09:55:44,881 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for e20e3543303de552ba42845d95488569 in 1661ms, sequenceid=13, compaction requested=true 2023-02-09 09:55:44,881 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e20e3543303de552ba42845d95488569: 2023-02-09 09:55:44,881 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:44,881 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:55:44,883 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/ab3af65f8bcd48258c9ecfd09ac620c1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/ab3af65f8bcd48258c9ecfd09ac620c1 2023-02-09 09:55:44,886 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/ab3af65f8bcd48258c9ecfd09ac620c1, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 09:55:44,887 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/5fad7861847f410886fe32271ae155dd as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/5fad7861847f410886fe32271ae155dd 2023-02-09 09:55:44,890 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/5fad7861847f410886fe32271ae155dd, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 09:55:44,891 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for e761366629ce85046adc8c312fb15374 in 1671ms, sequenceid=13, compaction requested=true 2023-02-09 09:55:44,891 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for e761366629ce85046adc8c312fb15374: 2023-02-09 09:55:44,891 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:55:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:55:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 09:55:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 09:55:44,894 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:44,894 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:44,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:44,894 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:44,894 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:44,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:44,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:44,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:44,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:44,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 09:55:44,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10c558c5[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:44,895 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 09:55:44,895 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:44,896 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:44,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:44,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:44,897 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 09:55:44,897 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 09:55:44,897 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:44,897 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,897 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:44,898 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,898 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 09:55:44,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:44,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:44,898 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:44,898 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 09:55:44,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:44,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:44,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:44,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:44,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:44,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:44,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 09:55:44,900 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 09:55:44,900 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:44,900 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:44,900 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 09:55:44,900 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:44,901 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:44,901 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:44,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:44,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:44,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:44,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:44,901 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:44,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:44,902 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:44,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:44,903 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:44,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:44,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,906 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,906 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:44,906 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:44,906 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:44,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 09:55:44,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:44,906 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:44,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:44,906 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:44,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:44,907 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:44,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 09:55:44,907 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:44,907 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:44,907 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:44,907 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:44,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:44,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:44,907 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:45,007 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:45,007 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:45,007 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:45,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:45,009 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:45,009 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:45,009 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:55:45,011 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:45,011 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:45,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:45,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:45,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 09:55:45,013 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 09:55:45,013 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:45,013 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 09:55:45,013 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 09:55:45,013 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:45,013 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:45,015 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:45,015 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:45,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:45,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:45,015 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:45,015 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:45,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:45,015 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:45,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:45,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:45,016 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 09:55:45,016 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 09:55:45,016 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 09:55:45,016 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 09:55:45,016 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:45,016 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 09:55:45,016 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:45,016 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:55:45,016 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. started... 2023-02-09 09:55:45,016 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:45,016 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:45,016 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e20e3543303de552ba42845d95488569 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 09:55:45,016 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:45,016 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. started... 2023-02-09 09:55:45,016 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. started... 2023-02-09 09:55:45,017 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing d1ea6fc17b9a3bc112723fe71be3a078 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 09:55:45,017 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing e761366629ce85046adc8c312fb15374 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 09:55:45,028 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209affad6a3452148f8b1ec1ac80f7670f0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209affad6a3452148f8b1ec1ac80f7670f0 2023-02-09 09:55:45,028 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020973291c34bcb64454993a950fca667a97 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023020973291c34bcb64454993a950fca667a97 2023-02-09 09:55:45,429 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209eeb2d8afcdb542bca2fcd0268f609403 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209eeb2d8afcdb542bca2fcd0268f609403 2023-02-09 09:55:45,432 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/676cb5095d264e0b9b151fd265ba8d5b 2023-02-09 09:55:45,433 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/333200dd23fb4a28a0ae8a9a14690870 2023-02-09 09:55:45,834 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/0c3a1cf9e121452d88326a0d78c4c38c 2023-02-09 09:55:45,845 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209e2359306f4f043dcbb1e1d458c56ed42 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230209e2359306f4f043dcbb1e1d458c56ed42 2023-02-09 09:55:45,845 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302092d17031d547649adb37efffbb7a53a40 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302092d17031d547649adb37efffbb7a53a40 2023-02-09 09:55:45,849 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302099059810725c94a68913316b4d66ed91e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302099059810725c94a68913316b4d66ed91e 2023-02-09 09:55:46,250 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/3ed7fb64748e4de5b0db6cab84b56f1a 2023-02-09 09:55:46,250 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/0873b415f129481d90d8f52510ce442a 2023-02-09 09:55:46,253 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/e639bd6d2f44489c9ed3e5fe18dd59fe 2023-02-09 09:55:46,254 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/333200dd23fb4a28a0ae8a9a14690870 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/333200dd23fb4a28a0ae8a9a14690870 2023-02-09 09:55:46,254 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/676cb5095d264e0b9b151fd265ba8d5b as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/676cb5095d264e0b9b151fd265ba8d5b 2023-02-09 09:55:46,257 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/0c3a1cf9e121452d88326a0d78c4c38c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/0c3a1cf9e121452d88326a0d78c4c38c 2023-02-09 09:55:46,258 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/333200dd23fb4a28a0ae8a9a14690870, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 09:55:46,258 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/676cb5095d264e0b9b151fd265ba8d5b, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 09:55:46,260 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/3ed7fb64748e4de5b0db6cab84b56f1a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/3ed7fb64748e4de5b0db6cab84b56f1a 2023-02-09 09:55:46,260 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/0873b415f129481d90d8f52510ce442a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/0873b415f129481d90d8f52510ce442a 2023-02-09 09:55:46,262 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/0c3a1cf9e121452d88326a0d78c4c38c, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 09:55:46,263 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/e639bd6d2f44489c9ed3e5fe18dd59fe as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/e639bd6d2f44489c9ed3e5fe18dd59fe 2023-02-09 09:55:46,263 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/3ed7fb64748e4de5b0db6cab84b56f1a, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 09:55:46,264 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/0873b415f129481d90d8f52510ce442a, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 09:55:46,264 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for e761366629ce85046adc8c312fb15374 in 1248ms, sequenceid=17, compaction requested=true 2023-02-09 09:55:46,264 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d1ea6fc17b9a3bc112723fe71be3a078 in 1248ms, sequenceid=17, compaction requested=true 2023-02-09 09:55:46,264 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for e761366629ce85046adc8c312fb15374: 2023-02-09 09:55:46,264 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for d1ea6fc17b9a3bc112723fe71be3a078: 2023-02-09 09:55:46,264 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:46,264 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:46,264 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:55:46,264 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:55:46,267 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/e639bd6d2f44489c9ed3e5fe18dd59fe, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 09:55:46,267 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for e20e3543303de552ba42845d95488569 in 1251ms, sequenceid=17, compaction requested=true 2023-02-09 09:55:46,267 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e20e3543303de552ba42845d95488569: 2023-02-09 09:55:46,267 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:46,267 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:55:46,267 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:55:46,267 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,267 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 09:55:46,267 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 09:55:46,269 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:46,269 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:46,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:46,269 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:46,269 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:46,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:46,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:46,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:46,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:46,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 09:55:46,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e54d80f[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:46,270 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 09:55:46,271 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:46,271 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:46,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:46,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:46,272 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 09:55:46,272 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 09:55:46,272 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:46,272 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,272 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:46,274 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 09:55:46,274 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,274 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:46,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:46,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:46,274 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 09:55:46,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:46,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:46,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:46,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:46,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:46,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:46,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 09:55:46,276 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 09:55:46,276 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:46,276 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:46,276 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 09:55:46,276 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:46,277 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:46,278 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:46,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:46,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:46,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:46,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:46,278 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:46,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:46,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:46,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:46,278 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:46,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:46,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:46,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:46,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:46,279 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:46,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:46,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,281 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,281 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:46,281 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:46,281 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,281 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:46,281 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 09:55:46,281 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:46,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 09:55:46,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:46,281 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:46,281 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:46,281 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:46,281 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:46,282 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:46,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:46,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:46,282 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:46,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:46,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:46,381 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:46,382 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:46,382 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:46,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:46,392 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:46,400 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:46,411 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:46,412 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:46,437 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:46,437 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:46,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:46,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:46,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 09:55:46,439 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 09:55:46,439 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:46,439 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 09:55:46,439 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 09:55:46,439 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:46,439 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:46,440 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,440 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:46,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:46,441 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:46,441 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:46,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:46,441 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 09:55:46,441 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 09:55:46,441 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 09:55:46,442 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 09:55:46,442 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:46,442 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 09:55:46,442 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:46,442 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. started... 2023-02-09 09:55:46,442 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e20e3543303de552ba42845d95488569 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 09:55:46,443 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:46,443 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. started... 2023-02-09 09:55:46,444 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:55:46,444 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing d1ea6fc17b9a3bc112723fe71be3a078 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 09:55:46,444 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:46,444 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. started... 2023-02-09 09:55:46,444 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:46,444 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing e761366629ce85046adc8c312fb15374 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 09:55:46,459 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/831534f533394ba4804532f8b04000f4 2023-02-09 09:55:46,462 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 831534f533394ba4804532f8b04000f4 2023-02-09 09:55:46,858 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/53424a2c03764f2aa9850c075c066878 2023-02-09 09:55:46,862 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 53424a2c03764f2aa9850c075c066878 2023-02-09 09:55:47,261 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/82a61c34a18d4cb681efef4da2c7c1b6 2023-02-09 09:55:47,265 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 82a61c34a18d4cb681efef4da2c7c1b6 2023-02-09 09:55:47,272 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/63f5d2855093418b903dc7db6588e26c 2023-02-09 09:55:47,274 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/33789ddcb8834d508bb0ec51b1df53e7 2023-02-09 09:55:47,275 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63f5d2855093418b903dc7db6588e26c 2023-02-09 09:55:47,276 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/831534f533394ba4804532f8b04000f4 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/831534f533394ba4804532f8b04000f4 2023-02-09 09:55:47,277 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33789ddcb8834d508bb0ec51b1df53e7 2023-02-09 09:55:47,278 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/53424a2c03764f2aa9850c075c066878 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/53424a2c03764f2aa9850c075c066878 2023-02-09 09:55:47,279 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 831534f533394ba4804532f8b04000f4 2023-02-09 09:55:47,279 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/831534f533394ba4804532f8b04000f4, entries=3, sequenceid=23, filesize=5.0 K 2023-02-09 09:55:47,280 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/63f5d2855093418b903dc7db6588e26c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/63f5d2855093418b903dc7db6588e26c 2023-02-09 09:55:47,281 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 53424a2c03764f2aa9850c075c066878 2023-02-09 09:55:47,281 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/53424a2c03764f2aa9850c075c066878, entries=3, sequenceid=23, filesize=5.0 K 2023-02-09 09:55:47,282 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/33789ddcb8834d508bb0ec51b1df53e7 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/33789ddcb8834d508bb0ec51b1df53e7 2023-02-09 09:55:47,283 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63f5d2855093418b903dc7db6588e26c 2023-02-09 09:55:47,284 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/63f5d2855093418b903dc7db6588e26c, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 09:55:47,284 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for e20e3543303de552ba42845d95488569 in 842ms, sequenceid=23, compaction requested=true 2023-02-09 09:55:47,284 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e20e3543303de552ba42845d95488569: 2023-02-09 09:55:47,284 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:47,284 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:55:47,286 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33789ddcb8834d508bb0ec51b1df53e7 2023-02-09 09:55:47,286 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/33789ddcb8834d508bb0ec51b1df53e7, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 09:55:47,286 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d1ea6fc17b9a3bc112723fe71be3a078 in 842ms, sequenceid=23, compaction requested=true 2023-02-09 09:55:47,286 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for d1ea6fc17b9a3bc112723fe71be3a078: 2023-02-09 09:55:47,286 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:47,286 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:55:48,077 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/006acd9289824d638ee5145025d7db62 2023-02-09 09:55:48,081 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 006acd9289824d638ee5145025d7db62 2023-02-09 09:55:48,082 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/82a61c34a18d4cb681efef4da2c7c1b6 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/82a61c34a18d4cb681efef4da2c7c1b6 2023-02-09 09:55:48,085 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 82a61c34a18d4cb681efef4da2c7c1b6 2023-02-09 09:55:48,086 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/82a61c34a18d4cb681efef4da2c7c1b6, entries=3, sequenceid=23, filesize=5.0 K 2023-02-09 09:55:48,086 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/006acd9289824d638ee5145025d7db62 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/006acd9289824d638ee5145025d7db62 2023-02-09 09:55:48,089 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 006acd9289824d638ee5145025d7db62 2023-02-09 09:55:48,090 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/006acd9289824d638ee5145025d7db62, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 09:55:48,090 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for e761366629ce85046adc8c312fb15374 in 1646ms, sequenceid=23, compaction requested=true 2023-02-09 09:55:48,090 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for e761366629ce85046adc8c312fb15374: 2023-02-09 09:55:48,090 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:48,090 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:55:48,091 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:55:48,091 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,091 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 09:55:48,091 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 09:55:48,093 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:48,093 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:48,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:48,093 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:48,093 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:48,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:48,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:48,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:48,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:48,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 09:55:48,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41722c7b[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:48,095 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 09:55:48,095 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:48,095 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:48,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:48,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:48,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 09:55:48,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 09:55:48,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:48,096 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,096 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:48,097 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 09:55:48,097 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:48,097 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,097 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 09:55:48,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:48,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:48,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:48,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:48,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:48,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:48,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:48,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:48,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 09:55:48,100 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 09:55:48,100 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:48,100 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:48,100 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 09:55:48,100 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:48,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:48,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:48,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:48,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:48,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:48,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:48,110 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:48,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:48,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:48,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:48,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:48,110 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:48,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:48,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:48,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:48,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,112 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:48,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 09:55:48,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,113 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:48,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 09:55:48,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:48,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:48,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 09:55:48,114 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:48,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:48,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 09:55:48,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 09:55:48,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:48,114 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:48,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:48,114 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:48,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 09:55:48,114 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:48,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:48,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:48,215 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:48,215 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:48,215 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:48,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:48,216 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:48,216 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): e20e3543303de552ba42845d95488569/family1 is initiating major compaction (all files) 2023-02-09 09:55:48,216 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:48,216 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:48,216 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of e20e3543303de552ba42845d95488569/family1 in ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:48,216 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/00b95c158ea64237b8b5beb648ab7e6c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/c9f92d88613043afae05fcb787bc3dc0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/deb9e58575f7460182e3eb47e6781301, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/0c3a1cf9e121452d88326a0d78c4c38c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/831534f533394ba4804532f8b04000f4] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp, totalSize=26.6 K 2023-02-09 09:55:48,217 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 00b95c158ea64237b8b5beb648ab7e6c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936540013 2023-02-09 09:55:48,217 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting c9f92d88613043afae05fcb787bc3dc0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936541414 2023-02-09 09:55:48,217 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting deb9e58575f7460182e3eb47e6781301, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936543213 2023-02-09 09:55:48,218 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 0c3a1cf9e121452d88326a0d78c4c38c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936545009 2023-02-09 09:55:48,218 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 831534f533394ba4804532f8b04000f4, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:55:48,226 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): e20e3543303de552ba42845d95488569#family1#compaction#149 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:48,634 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302096786df9afed14739b6962aebc1e5abe8_del 2023-02-09 09:55:48,634 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302096786df9afed14739b6962aebc1e5abe8_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302096786df9afed14739b6962aebc1e5abe8_del 2023-02-09 09:55:49,446 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b502bdc2bb548658bf649d2b227b3e2 2023-02-09 09:55:49,447 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family1/4b502bdc2bb548658bf649d2b227b3e2 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/4b502bdc2bb548658bf649d2b227b3e2 2023-02-09 09:55:49,450 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b502bdc2bb548658bf649d2b227b3e2 2023-02-09 09:55:49,451 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e20e3543303de552ba42845d95488569/family1 of e20e3543303de552ba42845d95488569 into 4b502bdc2bb548658bf649d2b227b3e2(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:49,451 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for e20e3543303de552ba42845d95488569: 2023-02-09 09:55:49,451 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:49,451 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): e20e3543303de552ba42845d95488569/family2 is initiating major compaction (all files) 2023-02-09 09:55:49,451 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:49,451 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:49,451 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of e20e3543303de552ba42845d95488569/family2 in ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:55:49,452 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/c6ece1a28565414a8a281770ba47ae85, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4c5c80582f734dfa81b3b137b3e57d25, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4a99bfe656a04c79910bbdd354966cff, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/e639bd6d2f44489c9ed3e5fe18dd59fe, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/63f5d2855093418b903dc7db6588e26c] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp, totalSize=25.3 K 2023-02-09 09:55:49,452 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting c6ece1a28565414a8a281770ba47ae85, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936540013 2023-02-09 09:55:49,452 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 4c5c80582f734dfa81b3b137b3e57d25, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936541414 2023-02-09 09:55:49,453 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 4a99bfe656a04c79910bbdd354966cff, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936543213 2023-02-09 09:55:49,453 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting e639bd6d2f44489c9ed3e5fe18dd59fe, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936545009 2023-02-09 09:55:49,453 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 63f5d2855093418b903dc7db6588e26c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:55:49,460 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): e20e3543303de552ba42845d95488569#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:49,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/00b95c158ea64237b8b5beb648ab7e6c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/c9f92d88613043afae05fcb787bc3dc0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/deb9e58575f7460182e3eb47e6781301, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/0c3a1cf9e121452d88326a0d78c4c38c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/831534f533394ba4804532f8b04000f4] to archive 2023-02-09 09:55:49,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:49,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/00b95c158ea64237b8b5beb648ab7e6c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/00b95c158ea64237b8b5beb648ab7e6c 2023-02-09 09:55:49,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/c9f92d88613043afae05fcb787bc3dc0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/c9f92d88613043afae05fcb787bc3dc0 2023-02-09 09:55:49,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/deb9e58575f7460182e3eb47e6781301 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/deb9e58575f7460182e3eb47e6781301 2023-02-09 09:55:49,479 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/0c3a1cf9e121452d88326a0d78c4c38c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/0c3a1cf9e121452d88326a0d78c4c38c 2023-02-09 09:55:49,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/831534f533394ba4804532f8b04000f4 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/831534f533394ba4804532f8b04000f4 2023-02-09 09:55:49,867 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209aa2c457b1b054423a063c9fad26dc0f7_del 2023-02-09 09:55:49,868 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209aa2c457b1b054423a063c9fad26dc0f7_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230209aa2c457b1b054423a063c9fad26dc0f7_del 2023-02-09 09:55:50,689 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a6b4e0ffb76e4246bd8f4d276074f80f 2023-02-09 09:55:50,689 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/.tmp/family2/a6b4e0ffb76e4246bd8f4d276074f80f as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/a6b4e0ffb76e4246bd8f4d276074f80f 2023-02-09 09:55:50,692 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a6b4e0ffb76e4246bd8f4d276074f80f 2023-02-09 09:55:50,693 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e20e3543303de552ba42845d95488569/family2 of e20e3543303de552ba42845d95488569 into a6b4e0ffb76e4246bd8f4d276074f80f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:50,693 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for e20e3543303de552ba42845d95488569: 2023-02-09 09:55:50,693 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:50,694 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): d1ea6fc17b9a3bc112723fe71be3a078/family1 is initiating major compaction (all files) 2023-02-09 09:55:50,694 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:50,694 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:50,694 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of d1ea6fc17b9a3bc112723fe71be3a078/family1 in ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:50,694 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/889da0fcb06a487c9f1db7052bc204f0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/13450e5d1e014992b4869db2e225ad98, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/ddf9c4bd83bb47269b0338fb177eb98e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/676cb5095d264e0b9b151fd265ba8d5b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/53424a2c03764f2aa9850c075c066878] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp, totalSize=26.6 K 2023-02-09 09:55:50,694 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 889da0fcb06a487c9f1db7052bc204f0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936540012 2023-02-09 09:55:50,695 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 13450e5d1e014992b4869db2e225ad98, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936541413 2023-02-09 09:55:50,695 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting ddf9c4bd83bb47269b0338fb177eb98e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936543213 2023-02-09 09:55:50,695 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 676cb5095d264e0b9b151fd265ba8d5b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936545009 2023-02-09 09:55:50,695 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 53424a2c03764f2aa9850c075c066878, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:55:50,702 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): d1ea6fc17b9a3bc112723fe71be3a078#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:50,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/c6ece1a28565414a8a281770ba47ae85, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4c5c80582f734dfa81b3b137b3e57d25, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4a99bfe656a04c79910bbdd354966cff, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/e639bd6d2f44489c9ed3e5fe18dd59fe, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/63f5d2855093418b903dc7db6588e26c] to archive 2023-02-09 09:55:50,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:50,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/c6ece1a28565414a8a281770ba47ae85 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/c6ece1a28565414a8a281770ba47ae85 2023-02-09 09:55:50,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4c5c80582f734dfa81b3b137b3e57d25 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4c5c80582f734dfa81b3b137b3e57d25 2023-02-09 09:55:50,782 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4a99bfe656a04c79910bbdd354966cff to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/4a99bfe656a04c79910bbdd354966cff 2023-02-09 09:55:50,783 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/e639bd6d2f44489c9ed3e5fe18dd59fe to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/e639bd6d2f44489c9ed3e5fe18dd59fe 2023-02-09 09:55:50,784 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/63f5d2855093418b903dc7db6588e26c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family2/63f5d2855093418b903dc7db6588e26c 2023-02-09 09:55:51,110 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209c7d2ab33aca04c6ea381fb91c3ebaa79_del 2023-02-09 09:55:51,110 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209c7d2ab33aca04c6ea381fb91c3ebaa79_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230209c7d2ab33aca04c6ea381fb91c3ebaa79_del 2023-02-09 09:55:51,930 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 221914da805144789138565e2eb0045e 2023-02-09 09:55:51,932 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family1/221914da805144789138565e2eb0045e as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/221914da805144789138565e2eb0045e 2023-02-09 09:55:51,936 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 221914da805144789138565e2eb0045e 2023-02-09 09:55:51,937 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d1ea6fc17b9a3bc112723fe71be3a078/family1 of d1ea6fc17b9a3bc112723fe71be3a078 into 221914da805144789138565e2eb0045e(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:51,937 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for d1ea6fc17b9a3bc112723fe71be3a078: 2023-02-09 09:55:51,938 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:51,938 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): d1ea6fc17b9a3bc112723fe71be3a078/family2 is initiating major compaction (all files) 2023-02-09 09:55:51,938 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:51,938 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:51,938 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of d1ea6fc17b9a3bc112723fe71be3a078/family2 in ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:55:51,938 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/ad7ea0bfc5274b198b80d02828cf2979, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/aedd485feb1944d2b3da52218930a973, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/f4c90badd95f40b2812cceb7aebe0547, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/0873b415f129481d90d8f52510ce442a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/33789ddcb8834d508bb0ec51b1df53e7] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp, totalSize=25.3 K 2023-02-09 09:55:51,939 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting ad7ea0bfc5274b198b80d02828cf2979, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936540012 2023-02-09 09:55:51,939 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting aedd485feb1944d2b3da52218930a973, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936541413 2023-02-09 09:55:51,939 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting f4c90badd95f40b2812cceb7aebe0547, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936543213 2023-02-09 09:55:51,940 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 0873b415f129481d90d8f52510ce442a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936545009 2023-02-09 09:55:51,940 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 33789ddcb8834d508bb0ec51b1df53e7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:55:51,954 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): d1ea6fc17b9a3bc112723fe71be3a078#family2#compaction#152 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:51,977 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/889da0fcb06a487c9f1db7052bc204f0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/13450e5d1e014992b4869db2e225ad98, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/ddf9c4bd83bb47269b0338fb177eb98e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/676cb5095d264e0b9b151fd265ba8d5b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/53424a2c03764f2aa9850c075c066878] to archive 2023-02-09 09:55:51,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:51,979 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/889da0fcb06a487c9f1db7052bc204f0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/889da0fcb06a487c9f1db7052bc204f0 2023-02-09 09:55:51,980 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/13450e5d1e014992b4869db2e225ad98 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/13450e5d1e014992b4869db2e225ad98 2023-02-09 09:55:51,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/ddf9c4bd83bb47269b0338fb177eb98e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/ddf9c4bd83bb47269b0338fb177eb98e 2023-02-09 09:55:51,982 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/676cb5095d264e0b9b151fd265ba8d5b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/676cb5095d264e0b9b151fd265ba8d5b 2023-02-09 09:55:51,983 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/53424a2c03764f2aa9850c075c066878 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/53424a2c03764f2aa9850c075c066878 2023-02-09 09:55:52,365 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023020990affb1da00f49a2a5de8e1b16105e7f_del 2023-02-09 09:55:52,365 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020990affb1da00f49a2a5de8e1b16105e7f_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023020990affb1da00f49a2a5de8e1b16105e7f_del 2023-02-09 09:55:53,178 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4912641075d64d7eb4873f4dcd2850d2 2023-02-09 09:55:53,179 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/.tmp/family2/4912641075d64d7eb4873f4dcd2850d2 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/4912641075d64d7eb4873f4dcd2850d2 2023-02-09 09:55:53,182 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4912641075d64d7eb4873f4dcd2850d2 2023-02-09 09:55:53,183 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d1ea6fc17b9a3bc112723fe71be3a078/family2 of d1ea6fc17b9a3bc112723fe71be3a078 into 4912641075d64d7eb4873f4dcd2850d2(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:55:53,183 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for d1ea6fc17b9a3bc112723fe71be3a078: 2023-02-09 09:55:53,184 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:53,184 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): e761366629ce85046adc8c312fb15374/family1 is initiating major compaction (all files) 2023-02-09 09:55:53,184 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:53,184 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:53,184 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of e761366629ce85046adc8c312fb15374/family1 in ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:53,184 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/69ef322672764861914b723a2974a804, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/e355d4d2ac65452884fb91b4ceabb476, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/ab3af65f8bcd48258c9ecfd09ac620c1, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/333200dd23fb4a28a0ae8a9a14690870, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/82a61c34a18d4cb681efef4da2c7c1b6] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp, totalSize=26.6 K 2023-02-09 09:55:53,184 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 69ef322672764861914b723a2974a804, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936540013 2023-02-09 09:55:53,185 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting e355d4d2ac65452884fb91b4ceabb476, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936541414 2023-02-09 09:55:53,185 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting ab3af65f8bcd48258c9ecfd09ac620c1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936543213 2023-02-09 09:55:53,185 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 333200dd23fb4a28a0ae8a9a14690870, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936545009 2023-02-09 09:55:53,186 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 82a61c34a18d4cb681efef4da2c7c1b6, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:55:53,195 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): e761366629ce85046adc8c312fb15374#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:53,279 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/ad7ea0bfc5274b198b80d02828cf2979, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/aedd485feb1944d2b3da52218930a973, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/f4c90badd95f40b2812cceb7aebe0547, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/0873b415f129481d90d8f52510ce442a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/33789ddcb8834d508bb0ec51b1df53e7] to archive 2023-02-09 09:55:53,280 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:53,282 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/ad7ea0bfc5274b198b80d02828cf2979 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/ad7ea0bfc5274b198b80d02828cf2979 2023-02-09 09:55:53,283 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/aedd485feb1944d2b3da52218930a973 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/aedd485feb1944d2b3da52218930a973 2023-02-09 09:55:53,284 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/f4c90badd95f40b2812cceb7aebe0547 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/f4c90badd95f40b2812cceb7aebe0547 2023-02-09 09:55:53,285 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/0873b415f129481d90d8f52510ce442a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/0873b415f129481d90d8f52510ce442a 2023-02-09 09:55:53,286 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/33789ddcb8834d508bb0ec51b1df53e7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family2/33789ddcb8834d508bb0ec51b1df53e7 2023-02-09 09:55:53,602 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209d3ff81a4409f4be5aea6405a8a6ce57d_del 2023-02-09 09:55:53,602 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209d3ff81a4409f4be5aea6405a8a6ce57d_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209d3ff81a4409f4be5aea6405a8a6ce57d_del 2023-02-09 09:55:54,015 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b0945abc15dd4784929f94404740ac4a 2023-02-09 09:55:54,016 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family1/b0945abc15dd4784929f94404740ac4a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/b0945abc15dd4784929f94404740ac4a 2023-02-09 09:55:54,020 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b0945abc15dd4784929f94404740ac4a 2023-02-09 09:55:54,021 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e761366629ce85046adc8c312fb15374/family1 of e761366629ce85046adc8c312fb15374 into b0945abc15dd4784929f94404740ac4a(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:55:54,021 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for e761366629ce85046adc8c312fb15374: 2023-02-09 09:55:54,021 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:55:54,021 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): e761366629ce85046adc8c312fb15374/family2 is initiating major compaction (all files) 2023-02-09 09:55:54,021 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:55:54,021 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:55:54,022 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of e761366629ce85046adc8c312fb15374/family2 in ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:55:54,022 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/57d2f1a0571a43cc972b61678a5de71a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/2a7a08fe30004da58fa5a39f16f03163, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/5fad7861847f410886fe32271ae155dd, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/3ed7fb64748e4de5b0db6cab84b56f1a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/006acd9289824d638ee5145025d7db62] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp, totalSize=25.3 K 2023-02-09 09:55:54,022 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 57d2f1a0571a43cc972b61678a5de71a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936540013 2023-02-09 09:55:54,023 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 2a7a08fe30004da58fa5a39f16f03163, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936541414 2023-02-09 09:55:54,023 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 5fad7861847f410886fe32271ae155dd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936543213 2023-02-09 09:55:54,024 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 3ed7fb64748e4de5b0db6cab84b56f1a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936545009 2023-02-09 09:55:54,024 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 006acd9289824d638ee5145025d7db62, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:55:54,040 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): e761366629ce85046adc8c312fb15374#family2#compaction#154 average throughput is 0.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:55:54,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/69ef322672764861914b723a2974a804, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/e355d4d2ac65452884fb91b4ceabb476, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/ab3af65f8bcd48258c9ecfd09ac620c1, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/333200dd23fb4a28a0ae8a9a14690870, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/82a61c34a18d4cb681efef4da2c7c1b6] to archive 2023-02-09 09:55:54,080 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:54,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/69ef322672764861914b723a2974a804 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/69ef322672764861914b723a2974a804 2023-02-09 09:55:54,082 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/e355d4d2ac65452884fb91b4ceabb476 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/e355d4d2ac65452884fb91b4ceabb476 2023-02-09 09:55:54,083 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/ab3af65f8bcd48258c9ecfd09ac620c1 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/ab3af65f8bcd48258c9ecfd09ac620c1 2023-02-09 09:55:54,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/333200dd23fb4a28a0ae8a9a14690870 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/333200dd23fb4a28a0ae8a9a14690870 2023-02-09 09:55:54,085 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/82a61c34a18d4cb681efef4da2c7c1b6 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/82a61c34a18d4cb681efef4da2c7c1b6 2023-02-09 09:55:54,458 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:55:54,458 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e586a95 to 127.0.0.1:64170 2023-02-09 09:55:54,458 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:55:54,459 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302094d30f5c65c9d45949125df841a8059bd_del 2023-02-09 09:55:54,460 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302094d30f5c65c9d45949125df841a8059bd_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302094d30f5c65c9d45949125df841a8059bd_del 2023-02-09 09:55:54,878 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df33a17e69b7402e9f77e0d2dff199a5 2023-02-09 09:55:54,879 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/.tmp/family2/df33a17e69b7402e9f77e0d2dff199a5 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/df33a17e69b7402e9f77e0d2dff199a5 2023-02-09 09:55:54,883 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df33a17e69b7402e9f77e0d2dff199a5 2023-02-09 09:55:54,884 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e761366629ce85046adc8c312fb15374/family2 of e761366629ce85046adc8c312fb15374 into df33a17e69b7402e9f77e0d2dff199a5(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:55:54,884 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for e761366629ce85046adc8c312fb15374: 2023-02-09 09:55:54,887 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:54,890 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:54,894 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:54,895 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:54,902 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 09:55:54,918 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-02-09 09:55:54,922 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209c7d2ab33aca04c6ea381fb91c3ebaa79_del 2023-02-09 09:55:54,925 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302096786df9afed14739b6962aebc1e5abe8_del 2023-02-09 09:55:54,930 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209d3ff81a4409f4be5aea6405a8a6ce57d_del 2023-02-09 09:55:54,930 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-02-09 09:55:54,931 DEBUG [Listener at localhost/42557] zookeeper.ReadOnlyZKClient(139): Connect 0x02465de9 to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:55:54,935 DEBUG [Listener at localhost/42557] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@68e83979, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:55:54,937 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 09:55:54,937 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 09:55:54,937 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 09:55:54,976 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/57d2f1a0571a43cc972b61678a5de71a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/2a7a08fe30004da58fa5a39f16f03163, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/5fad7861847f410886fe32271ae155dd, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/3ed7fb64748e4de5b0db6cab84b56f1a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/006acd9289824d638ee5145025d7db62] to archive 2023-02-09 09:55:54,976 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:54,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/57d2f1a0571a43cc972b61678a5de71a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/57d2f1a0571a43cc972b61678a5de71a 2023-02-09 09:55:54,980 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/2a7a08fe30004da58fa5a39f16f03163 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/2a7a08fe30004da58fa5a39f16f03163 2023-02-09 09:55:54,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/5fad7861847f410886fe32271ae155dd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/5fad7861847f410886fe32271ae155dd 2023-02-09 09:55:54,983 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/3ed7fb64748e4de5b0db6cab84b56f1a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/3ed7fb64748e4de5b0db6cab84b56f1a 2023-02-09 09:55:54,984 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/006acd9289824d638ee5145025d7db62 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family2/006acd9289824d638ee5145025d7db62 2023-02-09 09:55:55,400 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209ca08296f0ada4bdcbdc0b7f1caf85e92 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209ca08296f0ada4bdcbdc0b7f1caf85e92 2023-02-09 09:55:55,401 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 09:55:55,420 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:55:55,424 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36470, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:55:55,445 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:55,445 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:55,446 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:55:55,453 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50024, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:55:55,478 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5a93cf8045654b47a7fc9be984780d82 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-02-09 09:55:55,479 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5a93cf8045654b47a7fc9be984780d82}] 2023-02-09 09:55:55,486 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5a93cf8045654b47a7fc9be984780d82 for inclusion in e761366629ce85046adc8c312fb15374/family1 2023-02-09 09:55:55,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-02-09 09:55:55,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 09:55:55,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for e761366629ce85046adc8c312fb15374: 2023-02-09 09:55:55,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5a93cf8045654b47a7fc9be984780d82 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__li6igr766uudn7a65idvb7dp602kfe0d4aqpgluqcem54gnipnj826h0lhtnqu2/family1/5a93cf8045654b47a7fc9be984780d82 2023-02-09 09:55:55,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__li6igr766uudn7a65idvb7dp602kfe0d4aqpgluqcem54gnipnj826h0lhtnqu2/family1/5a93cf8045654b47a7fc9be984780d82 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/85c304067bae4599911a9201b66fb338_SeqId_28_ 2023-02-09 09:55:55,495 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__li6igr766uudn7a65idvb7dp602kfe0d4aqpgluqcem54gnipnj826h0lhtnqu2/family1/5a93cf8045654b47a7fc9be984780d82 into e761366629ce85046adc8c312fb15374/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/85c304067bae4599911a9201b66fb338_SeqId_28_ - updating store file list. 2023-02-09 09:55:55,502 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/85c304067bae4599911a9201b66fb338_SeqId_28_ into e761366629ce85046adc8c312fb15374/family1 2023-02-09 09:55:55,502 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__li6igr766uudn7a65idvb7dp602kfe0d4aqpgluqcem54gnipnj826h0lhtnqu2/family1/5a93cf8045654b47a7fc9be984780d82 into e761366629ce85046adc8c312fb15374/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/family1/85c304067bae4599911a9201b66fb338_SeqId_28_) 2023-02-09 09:55:55,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__li6igr766uudn7a65idvb7dp602kfe0d4aqpgluqcem54gnipnj826h0lhtnqu2/family1/5a93cf8045654b47a7fc9be984780d82 2023-02-09 09:55:55,510 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 09:55:55,511 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:55,513 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209e4bb4a937a4c404aa00e3a00509f2423 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209e4bb4a937a4c404aa00e3a00509f2423 2023-02-09 09:55:55,513 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-09 09:55:55,513 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 09:55:55,513 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 09:55:55,785 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209c4128089161b4147b7f1e2b79cf2aaed to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209c4128089161b4147b7f1e2b79cf2aaed 2023-02-09 09:55:55,786 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 09:55:55,792 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020947a5753e948f4855b0e90a4da63df371 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023020947a5753e948f4855b0e90a4da63df371 2023-02-09 09:55:55,793 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 09:55:55,818 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:55,818 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:55,824 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:55,825 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:55,841 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/95e8321598984bbe89d0c909689fde7c first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-02-09 09:55:55,843 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/95e8321598984bbe89d0c909689fde7c}] 2023-02-09 09:55:55,845 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/7a1574481f6845e0b1792640d11c0a38 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-02-09 09:55:55,846 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/95e8321598984bbe89d0c909689fde7c for inclusion in e20e3543303de552ba42845d95488569/family1 2023-02-09 09:55:55,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-02-09 09:55:55,848 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/7a1574481f6845e0b1792640d11c0a38}] 2023-02-09 09:55:55,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 09:55:55,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for e20e3543303de552ba42845d95488569: 2023-02-09 09:55:55,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/95e8321598984bbe89d0c909689fde7c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__a5b73chj73mkfaf44rkd53en19lcqmo09s9qdv8flmfpumqdckmgm4b11d4f17v6/family1/95e8321598984bbe89d0c909689fde7c 2023-02-09 09:55:55,852 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/7a1574481f6845e0b1792640d11c0a38 for inclusion in d1ea6fc17b9a3bc112723fe71be3a078/family1 2023-02-09 09:55:55,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-02-09 09:55:55,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 09:55:55,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for d1ea6fc17b9a3bc112723fe71be3a078: 2023-02-09 09:55:55,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/7a1574481f6845e0b1792640d11c0a38 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__ppgtjd9e7pcjlnmf54nslmcsc2od5lp4pne7tvvt8bj4crg56rheqtb53k3ffcfl/family1/7a1574481f6845e0b1792640d11c0a38 2023-02-09 09:55:55,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__a5b73chj73mkfaf44rkd53en19lcqmo09s9qdv8flmfpumqdckmgm4b11d4f17v6/family1/95e8321598984bbe89d0c909689fde7c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/ba7340546f704820acf978536a7e6b68_SeqId_28_ 2023-02-09 09:55:55,857 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__a5b73chj73mkfaf44rkd53en19lcqmo09s9qdv8flmfpumqdckmgm4b11d4f17v6/family1/95e8321598984bbe89d0c909689fde7c into e20e3543303de552ba42845d95488569/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/ba7340546f704820acf978536a7e6b68_SeqId_28_ - updating store file list. 2023-02-09 09:55:55,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__ppgtjd9e7pcjlnmf54nslmcsc2od5lp4pne7tvvt8bj4crg56rheqtb53k3ffcfl/family1/7a1574481f6845e0b1792640d11c0a38 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/0380a1f1bcd54dcfad511ced377eb564_SeqId_28_ 2023-02-09 09:55:55,860 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__ppgtjd9e7pcjlnmf54nslmcsc2od5lp4pne7tvvt8bj4crg56rheqtb53k3ffcfl/family1/7a1574481f6845e0b1792640d11c0a38 into d1ea6fc17b9a3bc112723fe71be3a078/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/0380a1f1bcd54dcfad511ced377eb564_SeqId_28_ - updating store file list. 2023-02-09 09:55:55,862 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/ba7340546f704820acf978536a7e6b68_SeqId_28_ into e20e3543303de552ba42845d95488569/family1 2023-02-09 09:55:55,862 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__a5b73chj73mkfaf44rkd53en19lcqmo09s9qdv8flmfpumqdckmgm4b11d4f17v6/family1/95e8321598984bbe89d0c909689fde7c into e20e3543303de552ba42845d95488569/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/family1/ba7340546f704820acf978536a7e6b68_SeqId_28_) 2023-02-09 09:55:55,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__a5b73chj73mkfaf44rkd53en19lcqmo09s9qdv8flmfpumqdckmgm4b11d4f17v6/family1/95e8321598984bbe89d0c909689fde7c 2023-02-09 09:55:55,865 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 09:55:55,866 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:55,867 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/0380a1f1bcd54dcfad511ced377eb564_SeqId_28_ into d1ea6fc17b9a3bc112723fe71be3a078/family1 2023-02-09 09:55:55,867 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__ppgtjd9e7pcjlnmf54nslmcsc2od5lp4pne7tvvt8bj4crg56rheqtb53k3ffcfl/family1/7a1574481f6845e0b1792640d11c0a38 into d1ea6fc17b9a3bc112723fe71be3a078/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/family1/0380a1f1bcd54dcfad511ced377eb564_SeqId_28_) 2023-02-09 09:55:55,867 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209ab8b43afc1234d91ac391e703e7b9efc to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209ab8b43afc1234d91ac391e703e7b9efc 2023-02-09 09:55:55,867 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-09 09:55:55,867 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 09:55:55,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__ns_testMinorCompaction__ppgtjd9e7pcjlnmf54nslmcsc2od5lp4pne7tvvt8bj4crg56rheqtb53k3ffcfl/family1/7a1574481f6845e0b1792640d11c0a38 2023-02-09 09:55:55,868 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 09:55:55,870 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 09:55:55,871 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:55:55,872 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023020982a8bf3cea684043908fe83939a92f7e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023020982a8bf3cea684043908fe83939a92f7e 2023-02-09 09:55:55,872 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-09 09:55:55,872 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 09:55:55,873 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 09:55:55,873 INFO [Listener at localhost/42557] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-02-09 09:55:55,875 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:55,878 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:55,881 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:55:55,882 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:55:55,899 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=251 (was 280), OpenFileDescriptor=637 (was 746), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=112 (was 111) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3837 (was 3240) - AvailableMemoryMB LEAK? - 2023-02-09 09:55:55,911 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=251, OpenFileDescriptor=637, MaxFileDescriptor=60000, SystemLoadAverage=112, ProcessCount=168, AvailableMemoryMB=3837 2023-02-09 09:55:55,912 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:55,912 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:55,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 09:55:55,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:55,916 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:55:55,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-02-09 09:55:55,916 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:55,916 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:55:55,916 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:55,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 09:55:55,917 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f 2023-02-09 09:55:55,918 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f empty. 2023-02-09 09:55:55,918 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f 2023-02-09 09:55:55,918 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-02-09 09:55:55,924 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 09:55:55,925 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2e714ed6a371d3ab3e132b9b4cdecb8f, NAME => 'testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:55:56,017 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:56,017 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:56,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 09:55:56,218 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:56,219 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:56,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 09:55:56,334 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:56,334 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 2e714ed6a371d3ab3e132b9b4cdecb8f, disabling compactions & flushes 2023-02-09 09:55:56,334 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:56,334 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:56,334 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. after waiting 0 ms 2023-02-09 09:55:56,334 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:56,334 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:56,334 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: Waiting for close lock at 1675936556334Disabling compacts and flushes for region at 1675936556334Disabling writes for close at 1675936556334Writing region close event to WAL at 1675936556334Closed at 1675936556334 2023-02-09 09:55:56,336 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:55:56,337 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675936556337"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936556337"}]},"ts":"1675936556337"} 2023-02-09 09:55:56,338 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 09:55:56,339 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:55:56,339 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936556339"}]},"ts":"1675936556339"} 2023-02-09 09:55:56,340 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-09 09:55:56,344 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2e714ed6a371d3ab3e132b9b4cdecb8f, ASSIGN}] 2023-02-09 09:55:56,345 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2e714ed6a371d3ab3e132b9b4cdecb8f, ASSIGN 2023-02-09 09:55:56,346 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2e714ed6a371d3ab3e132b9b4cdecb8f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:55:56,497 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2e714ed6a371d3ab3e132b9b4cdecb8f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:56,497 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675936556497"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936556497"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936556497"}]},"ts":"1675936556497"} 2023-02-09 09:55:56,499 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 2e714ed6a371d3ab3e132b9b4cdecb8f, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:55:56,520 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:56,520 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:56,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 09:55:56,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:56,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2e714ed6a371d3ab3e132b9b4cdecb8f, NAME => 'testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f.', STARTKEY => '', ENDKEY => ''} 2023-02-09 09:55:56,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 2e714ed6a371d3ab3e132b9b4cdecb8f 2023-02-09 09:55:56,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:55:56,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2e714ed6a371d3ab3e132b9b4cdecb8f 2023-02-09 09:55:56,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2e714ed6a371d3ab3e132b9b4cdecb8f 2023-02-09 09:55:56,655 INFO [StoreOpener-2e714ed6a371d3ab3e132b9b4cdecb8f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2e714ed6a371d3ab3e132b9b4cdecb8f 2023-02-09 09:55:56,656 DEBUG [StoreOpener-2e714ed6a371d3ab3e132b9b4cdecb8f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1 2023-02-09 09:55:56,656 DEBUG [StoreOpener-2e714ed6a371d3ab3e132b9b4cdecb8f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1 2023-02-09 09:55:56,657 INFO [StoreOpener-2e714ed6a371d3ab3e132b9b4cdecb8f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2e714ed6a371d3ab3e132b9b4cdecb8f columnFamilyName family1 2023-02-09 09:55:56,658 INFO [StoreOpener-2e714ed6a371d3ab3e132b9b4cdecb8f-1] regionserver.HStore(310): Store=2e714ed6a371d3ab3e132b9b4cdecb8f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:55:56,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f 2023-02-09 09:55:56,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f 2023-02-09 09:55:56,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2e714ed6a371d3ab3e132b9b4cdecb8f 2023-02-09 09:55:56,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:55:56,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2e714ed6a371d3ab3e132b9b4cdecb8f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60516048, jitterRate=-0.09824061393737793}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 09:55:56,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: Running coprocessor pre-open hook at 1675936556654Writing region info on filesystem at 1675936556654Initializing all the Stores at 1675936556654Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675936556654Cleaning up temporary data from old regions at 1675936556660 (+6 ms)Cleaning up detritus from prior splits at 1675936556660Running coprocessor post-open hooks at 1675936556663 (+3 ms)Region opened successfully at 1675936556663 2023-02-09 09:55:56,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f., pid=114, masterSystemTime=1675936556650 2023-02-09 09:55:56,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:56,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:56,666 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2e714ed6a371d3ab3e132b9b4cdecb8f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:56,666 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675936556666"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936556666"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936556666"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936556666"}]},"ts":"1675936556666"} 2023-02-09 09:55:56,669 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-09 09:55:56,669 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 2e714ed6a371d3ab3e132b9b4cdecb8f, server=jenkins-hbase4.apache.org,42719,1675936416061 in 168 msec 2023-02-09 09:55:56,670 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-09 09:55:56,670 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2e714ed6a371d3ab3e132b9b4cdecb8f, ASSIGN in 325 msec 2023-02-09 09:55:56,670 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:55:56,671 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936556671"}]},"ts":"1675936556671"} 2023-02-09 09:55:56,672 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-09 09:55:56,674 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:55:56,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 760 msec 2023-02-09 09:55:57,021 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:57,021 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:57,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 09:55:57,023 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-02-09 09:55:57,043 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:57,043 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:57,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:57,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,048 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:55:57,048 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:57,048 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:55:57,048 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:55:57,048 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,049 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,050 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,050 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:57,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:57,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:57,050 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,050 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:57,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:57,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,051 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:55:57,051 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:57,051 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:55:57,051 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:55:57,051 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:57,052 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:57,052 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. started... 2023-02-09 09:55:57,052 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e714ed6a371d3ab3e132b9b4cdecb8f 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 09:55:57,483 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f4a4acc80ffa47019e143340b55f0e22 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130f4a4acc80ffa47019e143340b55f0e22 2023-02-09 09:55:57,886 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/b523fc00d81d411080bdf1d88c96b492 2023-02-09 09:55:57,890 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/b523fc00d81d411080bdf1d88c96b492 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/b523fc00d81d411080bdf1d88c96b492 2023-02-09 09:55:57,893 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/b523fc00d81d411080bdf1d88c96b492, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-09 09:55:57,894 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2e714ed6a371d3ab3e132b9b4cdecb8f in 842ms, sequenceid=5, compaction requested=false 2023-02-09 09:55:57,894 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:55:57,895 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:55:57,895 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:57,895 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:55:57,895 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:55:57,895 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,895 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:55:57,895 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:55:57,897 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,897 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:57,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:57,897 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,897 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:57,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:57,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:57,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:57,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:55:57,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47f1963a[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:57,899 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:55:57,899 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,900 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,900 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:55:57,900 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:55:57,900 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:57,901 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,901 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:57,902 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:55:57,902 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,902 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:57,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:57,902 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:55:57,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:57,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:57,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:57,904 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:55:57,904 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:55:57,904 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:57,904 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:57,904 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,904 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:57,906 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,906 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:57,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,906 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:57,907 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,908 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:55:57,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:57,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:57,910 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:57,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,910 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:57,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:57,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:58,010 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:58,011 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:58,011 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:58,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:58,014 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:58,015 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:58,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:58,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:58,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,016 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:55:58,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:58,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:55:58,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:55:58,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,019 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,020 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:58,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:58,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:58,020 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,020 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:58,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,020 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:58,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,021 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,021 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:55:58,021 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,021 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:55:58,021 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:58,021 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:55:58,021 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:55:58,021 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:58,021 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:58,021 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. started... 2023-02-09 09:55:58,021 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e714ed6a371d3ab3e132b9b4cdecb8f 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 09:55:58,433 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112873b115cc855e469ab8b0c5ad8d8d8752 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112873b115cc855e469ab8b0c5ad8d8d8752 2023-02-09 09:55:58,838 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/9ec888a27b674bbcbeb22fc8bfc3689b 2023-02-09 09:55:58,843 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/9ec888a27b674bbcbeb22fc8bfc3689b as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/9ec888a27b674bbcbeb22fc8bfc3689b 2023-02-09 09:55:58,846 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/9ec888a27b674bbcbeb22fc8bfc3689b, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 09:55:58,847 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2e714ed6a371d3ab3e132b9b4cdecb8f in 826ms, sequenceid=9, compaction requested=false 2023-02-09 09:55:58,847 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:55:58,847 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:58,847 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:55:58,848 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:55:58,848 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,848 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:55:58,848 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:55:58,849 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,849 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:58,850 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,850 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:58,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:55:58,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17307b4b[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:58,851 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:55:58,851 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,852 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,852 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:55:58,852 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:55:58,852 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:58,852 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,852 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:58,853 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:55:58,853 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:58,853 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,853 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:55:58,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:58,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:58,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:55:58,856 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:55:58,856 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:58,856 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:58,856 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,856 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:58,862 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,862 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:58,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,862 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:58,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:58,863 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:58,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,864 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,866 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,866 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,866 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,866 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,866 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:58,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:55:58,866 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:58,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:55:58,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:58,866 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,866 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:58,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:58,867 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:58,867 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:58,867 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,867 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:58,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:58,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:58,967 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:58,967 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:58,967 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:58,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:58,972 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:58,972 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:58,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:58,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:58,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,974 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:55:58,974 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:58,974 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:55:58,974 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:55:58,974 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,975 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,979 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,979 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:58,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:58,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:58,980 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:58,980 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:58,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:58,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,981 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,981 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:55:58,981 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:58,981 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:55:58,981 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:58,981 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:55:58,981 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:55:58,981 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:58,981 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:58,981 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. started... 2023-02-09 09:55:58,981 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e714ed6a371d3ab3e132b9b4cdecb8f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:55:58,997 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201460f6b634ab04163bb2cb374577ead96 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201460f6b634ab04163bb2cb374577ead96 2023-02-09 09:55:59,005 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/99f4f88d3868413ebcf823d091ee7705 2023-02-09 09:55:59,009 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/99f4f88d3868413ebcf823d091ee7705 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/99f4f88d3868413ebcf823d091ee7705 2023-02-09 09:55:59,012 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/99f4f88d3868413ebcf823d091ee7705, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 09:55:59,013 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2e714ed6a371d3ab3e132b9b4cdecb8f in 32ms, sequenceid=13, compaction requested=true 2023-02-09 09:55:59,013 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:55:59,013 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:59,013 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:55:59,013 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:55:59,013 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,013 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:55:59,013 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:55:59,015 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,015 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:59,015 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,015 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:59,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:55:59,017 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34849024[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:59,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:55:59,017 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,018 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,018 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:55:59,018 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,018 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:59,018 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:55:59,018 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:59,019 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:55:59,019 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,019 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:59,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,019 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:55:59,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:59,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:59,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:59,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:55:59,021 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:55:59,021 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:59,021 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:59,021 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,021 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:59,024 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,024 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,024 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:59,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:59,024 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:59,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:59,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,025 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:59,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:59,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:55:59,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:59,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:59,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:59,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:59,027 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,027 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:59,028 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,028 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:59,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:55:59,028 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:59,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:59,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:59,128 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:59,128 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:59,128 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:59,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:59,132 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:59,132 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:59,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:59,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:59,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,134 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:55:59,134 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:59,134 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:55:59,135 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:55:59,135 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,135 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,136 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,136 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:59,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:59,136 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,136 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,137 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:55:59,137 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:55:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:55:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:55:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:59,138 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:59,138 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. started... 2023-02-09 09:55:59,138 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e714ed6a371d3ab3e132b9b4cdecb8f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:55:59,549 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057368e036084840699eea530672310cdd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512057368e036084840699eea530672310cdd 2023-02-09 09:55:59,554 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/89b7b91027a4478089bd339b4d4bc1f7 2023-02-09 09:55:59,557 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/89b7b91027a4478089bd339b4d4bc1f7 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/89b7b91027a4478089bd339b4d4bc1f7 2023-02-09 09:55:59,561 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/89b7b91027a4478089bd339b4d4bc1f7, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 09:55:59,562 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2e714ed6a371d3ab3e132b9b4cdecb8f in 424ms, sequenceid=17, compaction requested=true 2023-02-09 09:55:59,562 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:55:59,562 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:59,562 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:55:59,562 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:55:59,562 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,562 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:55:59,562 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:55:59,563 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,563 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:59,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:59,564 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:59,564 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:55:59,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:59,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:59,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:55:59,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66e02557[Count = 0] remaining members to acquire global barrier 2023-02-09 09:55:59,565 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:55:59,565 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,567 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,568 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:55:59,568 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:55:59,568 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:55:59,568 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,568 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:55:59,569 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:55:59,569 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,569 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:59,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:59,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:59,569 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:55:59,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:59,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:59,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:59,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:55:59,571 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:55:59,571 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:55:59,571 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:55:59,571 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:55:59,571 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,571 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:55:59,573 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,573 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:55:59,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,573 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:55:59,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:55:59,573 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:55:59,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,575 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:55:59,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,577 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:59,577 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,577 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:59,577 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:55:59,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:55:59,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:55:59,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:59,577 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,577 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:55:59,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:55:59,578 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,578 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,578 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:55:59,578 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:55:59,578 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,578 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:55:59,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:55:59,678 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:55:59,678 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:59,678 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:59,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:55:59,682 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:55:59,682 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:55:59,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:55:59,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:55:59,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,684 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:55:59,684 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:59,684 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:55:59,684 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:55:59,685 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,685 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,687 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,687 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:55:59,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:55:59,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:55:59,687 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:55:59,687 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:55:59,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:55:59,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,688 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:55:59,688 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:55:59,688 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:55:59,688 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:55:59,688 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:55:59,688 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:55:59,688 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:55:59,688 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:55:59,689 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. started... 2023-02-09 09:55:59,689 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e714ed6a371d3ab3e132b9b4cdecb8f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:56:00,100 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228903cb30a18be45f3a6ceece4d5b0c8d7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228903cb30a18be45f3a6ceece4d5b0c8d7 2023-02-09 09:56:00,508 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/d9c69fd270774e49a0408b7c9da88356 2023-02-09 09:56:00,512 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/d9c69fd270774e49a0408b7c9da88356 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/d9c69fd270774e49a0408b7c9da88356 2023-02-09 09:56:00,515 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/d9c69fd270774e49a0408b7c9da88356, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 09:56:00,516 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2e714ed6a371d3ab3e132b9b4cdecb8f in 827ms, sequenceid=21, compaction requested=true 2023-02-09 09:56:00,516 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:00,516 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:00,516 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:56:00,516 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:56:00,516 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,516 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:56:00,516 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:56:00,517 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,517 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:00,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:00,518 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,518 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:00,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:00,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:00,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:00,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:56:00,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d6b2ff7[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:00,519 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:56:00,519 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,521 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,521 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:56:00,521 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:56:00,521 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:00,521 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,521 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:00,522 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,522 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:56:00,522 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:00,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,522 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:56:00,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:00,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:00,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:56:00,524 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:56:00,524 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:00,524 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:00,524 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,524 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:00,527 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,527 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:00,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,527 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:00,528 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,529 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,533 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,533 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:00,533 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,533 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:00,533 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:00,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:00,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:00,533 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,533 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,533 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,533 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,533 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,533 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:56:00,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:00,533 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:00,534 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:00,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:00,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:00,634 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:00,634 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:00,634 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:00,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:00,638 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:00,638 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:00,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:56:00,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:56:00,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,643 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:00,643 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:00,643 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:56:00,643 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:56:00,644 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,644 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,645 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,645 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:00,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:00,645 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:00,645 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:56:00,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:56:00,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,646 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,646 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:56:00,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:00,646 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:56:00,647 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:00,647 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:56:00,647 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:56:00,647 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:56:00,647 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:00,647 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. started... 2023-02-09 09:56:00,647 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e714ed6a371d3ab3e132b9b4cdecb8f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:56:01,058 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123107cb787d7bb8423fbb4743e5a199b896 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123107cb787d7bb8423fbb4743e5a199b896 2023-02-09 09:56:01,463 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/654750f0de9d494cbac9afcb59e40fb7 2023-02-09 09:56:01,467 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/654750f0de9d494cbac9afcb59e40fb7 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/654750f0de9d494cbac9afcb59e40fb7 2023-02-09 09:56:01,470 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/654750f0de9d494cbac9afcb59e40fb7, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 09:56:01,471 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2e714ed6a371d3ab3e132b9b4cdecb8f in 824ms, sequenceid=25, compaction requested=true 2023-02-09 09:56:01,471 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:01,471 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:01,471 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:56:01,471 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:56:01,471 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,471 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:56:01,471 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:56:01,473 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,473 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:01,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:01,473 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,473 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:01,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:01,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:01,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:01,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:56:01,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3995f332[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:01,475 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:56:01,475 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,477 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:56:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:56:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:01,478 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,478 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:01,479 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:56:01,479 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,479 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:01,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:01,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:01,479 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:56:01,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:01,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:01,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:01,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:01,481 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:56:01,481 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:56:01,481 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:01,481 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:01,481 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,481 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:01,482 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,482 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:01,482 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:01,483 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:01,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,484 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:01,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:01,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:01,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:01,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:01,486 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 09:56:01,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:01,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,486 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:01,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:01,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,486 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:01,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:01,487 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:01,487 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:01,587 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:01,587 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:01,587 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:01,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:01,591 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:01,591 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:01,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:56:01,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:56:01,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,593 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:01,593 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:01,594 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:56:01,594 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:56:01,594 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,594 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,596 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,596 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:01,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:01,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:01,597 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:01,597 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:56:01,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:56:01,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,597 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:56:01,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:01,598 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:56:01,598 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:01,598 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:56:01,598 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:56:01,598 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:56:01,598 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:01,598 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. started... 2023-02-09 09:56:01,598 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e714ed6a371d3ab3e132b9b4cdecb8f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:56:01,608 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601017b0126274f1c46dd89291e8657554e26 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601017b0126274f1c46dd89291e8657554e26 2023-02-09 09:56:02,013 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/c81c5181a96542bdba278eb6c995f674 2023-02-09 09:56:02,017 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/c81c5181a96542bdba278eb6c995f674 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/c81c5181a96542bdba278eb6c995f674 2023-02-09 09:56:02,020 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/c81c5181a96542bdba278eb6c995f674, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 09:56:02,020 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2e714ed6a371d3ab3e132b9b4cdecb8f in 422ms, sequenceid=29, compaction requested=true 2023-02-09 09:56:02,020 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:02,020 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:02,020 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:56:02,021 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:56:02,021 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,021 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:56:02,021 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:56:02,022 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,022 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:02,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:02,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:02,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:02,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:02,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:56:02,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29805194[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:02,024 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:56:02,024 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,025 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,025 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:56:02,025 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:56:02,025 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:02,025 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,025 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:56:02,026 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:02,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:02,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:56:02,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:02,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:02,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:02,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:02,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:56:02,029 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:56:02,029 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:02,029 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:02,029 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,029 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:02,031 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,031 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:02,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,031 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:02,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:02,032 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,033 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:02,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:02,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:02,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:02,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,035 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,035 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:02,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:02,135 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:02,136 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:02,136 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:02,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:02,140 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:02,140 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:02,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:56:02,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:56:02,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,142 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:02,142 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:02,142 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:56:02,142 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:56:02,142 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,142 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,143 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:02,143 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:02,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:02,144 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,144 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:56:02,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:56:02,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:56:02,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,145 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:56:02,145 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:02,145 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:56:02,145 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:56:02,145 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:56:02,145 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:02,145 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. started... 2023-02-09 09:56:02,145 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e714ed6a371d3ab3e132b9b4cdecb8f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:56:02,189 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:56:02,555 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010301e14f162cb44574b620c0d5c3599554 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010301e14f162cb44574b620c0d5c3599554 2023-02-09 09:56:02,960 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/6a8a63a4e22a4046be24bdee6cb8026a 2023-02-09 09:56:02,964 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/6a8a63a4e22a4046be24bdee6cb8026a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/6a8a63a4e22a4046be24bdee6cb8026a 2023-02-09 09:56:02,967 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/6a8a63a4e22a4046be24bdee6cb8026a, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 09:56:02,968 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2e714ed6a371d3ab3e132b9b4cdecb8f in 823ms, sequenceid=33, compaction requested=true 2023-02-09 09:56:02,968 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:02,968 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:02,968 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:56:02,968 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:56:02,968 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,968 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:56:02,968 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:56:02,970 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,970 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:02,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:02,970 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,970 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:02,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:02,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:56:02,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e2d136e[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:02,971 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:56:02,972 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,973 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,974 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:56:02,974 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:56:02,974 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:02,974 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,974 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:02,975 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:56:02,975 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,975 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:02,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:02,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:02,975 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:56:02,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:02,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:02,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:02,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:02,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:56:02,977 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:56:02,977 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:02,977 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:02,977 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,977 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:02,978 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,978 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:02,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,979 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:02,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:02,979 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:02,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,980 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:02,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:02,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:02,985 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:02,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:02,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:02,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:02,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:02,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:02,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,985 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:02,986 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:02,986 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:02,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:02,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:03,086 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:03,086 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:03,086 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:03,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:03,090 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:03,090 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:03,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:56:03,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:56:03,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,092 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:03,092 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:03,092 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:56:03,092 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:56:03,093 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,093 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,094 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:03,094 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:03,094 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,094 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,095 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:56:03,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:03,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:56:03,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:56:03,096 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:56:03,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:03,096 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. started... 2023-02-09 09:56:03,096 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e714ed6a371d3ab3e132b9b4cdecb8f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:56:03,507 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509074c1c200518194e57bb7e6c0ed17ce7e5 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201509074c1c200518194e57bb7e6c0ed17ce7e5 2023-02-09 09:56:03,913 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/5c14574da15e40a789e02dc0cef95c7a 2023-02-09 09:56:03,917 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/5c14574da15e40a789e02dc0cef95c7a as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/5c14574da15e40a789e02dc0cef95c7a 2023-02-09 09:56:03,920 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/5c14574da15e40a789e02dc0cef95c7a, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 09:56:03,921 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2e714ed6a371d3ab3e132b9b4cdecb8f in 825ms, sequenceid=37, compaction requested=true 2023-02-09 09:56:03,921 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:03,921 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:03,921 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:56:03,921 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:56:03,921 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,921 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:56:03,921 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:56:03,922 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,922 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:03,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:03,923 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,923 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:03,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:03,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:03,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:03,924 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:56:03,924 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3acd89cb[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:03,924 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:56:03,924 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,926 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,927 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:56:03,927 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:56:03,927 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:03,927 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,927 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:03,928 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:56:03,928 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,928 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:03,928 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:56:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:03,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:56:03,930 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:56:03,930 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:03,930 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:03,930 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,930 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:03,931 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,931 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:03,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,931 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:03,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:03,932 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,933 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,935 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,935 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:03,935 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:03,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:03,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:03,935 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,935 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:03,935 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:03,935 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:03,935 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,935 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,935 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:03,935 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:03,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:03,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:03,936 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:04,036 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:04,036 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:04,036 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:04,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:04,040 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:04,040 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:04,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:56:04,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 09:56:04,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,042 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:04,042 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:04,042 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 09:56:04,042 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 09:56:04,043 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,043 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,044 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:04,044 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:04,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:04,044 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,044 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:56:04,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:56:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,045 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,045 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 09:56:04,045 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,045 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 09:56:04,045 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:04,045 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 09:56:04,045 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 09:56:04,045 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:56:04,045 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:04,045 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. started... 2023-02-09 09:56:04,045 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2e714ed6a371d3ab3e132b9b4cdecb8f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 09:56:04,456 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120032d972a4b75408d8bc002a60b241b50 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120032d972a4b75408d8bc002a60b241b50 2023-02-09 09:56:04,861 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/a5754e5742c24947b7871e2969eaec80 2023-02-09 09:56:04,865 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/a5754e5742c24947b7871e2969eaec80 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/a5754e5742c24947b7871e2969eaec80 2023-02-09 09:56:04,868 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/a5754e5742c24947b7871e2969eaec80, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 09:56:04,869 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2e714ed6a371d3ab3e132b9b4cdecb8f in 824ms, sequenceid=41, compaction requested=true 2023-02-09 09:56:04,869 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:04,869 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:04,869 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 09:56:04,869 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 09:56:04,869 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,870 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 09:56:04,870 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 09:56:04,871 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,871 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:04,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:04,871 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,871 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:04,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:04,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:04,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:04,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 09:56:04,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2cd7e1c3[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:04,872 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 09:56:04,873 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,873 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,874 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,874 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:04,874 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 09:56:04,874 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 09:56:04,874 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:04,876 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,876 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 09:56:04,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:04,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:04,876 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:04,877 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 09:56:04,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:04,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:04,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:04,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:04,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 09:56:04,878 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 09:56:04,878 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:04,878 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:04,879 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,879 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:04,880 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:04,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:04,880 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,880 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:04,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:04,880 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:04,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:04,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,881 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,883 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,883 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:04,883 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,883 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:04,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:04,883 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:04,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:04,883 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 09:56:04,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:04,883 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:04,884 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:04,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:04,884 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,884 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,884 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:04,884 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:04,984 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:04,984 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:04,984 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:04,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:04,986 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-02-09 09:56:04,987 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:56:04,988 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:56:04,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:56:04,989 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937164989 2023-02-09 09:56:04,990 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 09:56:04,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-09 09:56:04,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:56:04,991 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x2728fdbe to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:56:04,995 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3ed537aa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:56:04,996 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:56:05,822 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036dcef7b6da9d40a8964f2fcdb19fc288 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601036dcef7b6da9d40a8964f2fcdb19fc288 2023-02-09 09:56:05,823 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:56:05,836 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:56:05,838 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51960, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:56:05,843 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:05,843 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:05,844 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:56:05,845 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50844, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:56:05,852 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/00ec212e58f34dcc9aca52b572620f56 first=Optional[r5] last=Optional[r6] 2023-02-09 09:56:05,853 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/00ec212e58f34dcc9aca52b572620f56}] 2023-02-09 09:56:05,868 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/00ec212e58f34dcc9aca52b572620f56 for inclusion in 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 2023-02-09 09:56:05,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-09 09:56:05,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:56:05,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:05,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/00ec212e58f34dcc9aca52b572620f56 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__42e13i06kujip1km631qqdon6nb0hqn7et28h9cbbbnkjjl84jhijj9srlfdi9qq/family1/00ec212e58f34dcc9aca52b572620f56 2023-02-09 09:56:05,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__42e13i06kujip1km631qqdon6nb0hqn7et28h9cbbbnkjjl84jhijj9srlfdi9qq/family1/00ec212e58f34dcc9aca52b572620f56 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/42bac1fb639a4f628a31bf6f40cecd73_SeqId_44_ 2023-02-09 09:56:05,874 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__42e13i06kujip1km631qqdon6nb0hqn7et28h9cbbbnkjjl84jhijj9srlfdi9qq/family1/00ec212e58f34dcc9aca52b572620f56 into 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/42bac1fb639a4f628a31bf6f40cecd73_SeqId_44_ - updating store file list. 2023-02-09 09:56:05,877 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/42bac1fb639a4f628a31bf6f40cecd73_SeqId_44_ into 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 2023-02-09 09:56:05,877 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__42e13i06kujip1km631qqdon6nb0hqn7et28h9cbbbnkjjl84jhijj9srlfdi9qq/family1/00ec212e58f34dcc9aca52b572620f56 into 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/42bac1fb639a4f628a31bf6f40cecd73_SeqId_44_) 2023-02-09 09:56:05,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__42e13i06kujip1km631qqdon6nb0hqn7et28h9cbbbnkjjl84jhijj9srlfdi9qq/family1/00ec212e58f34dcc9aca52b572620f56 2023-02-09 09:56:05,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 09:56:05,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 09:56:05,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(1912): 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 is initiating minor compaction 2023-02-09 09:56:05,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@53801ae; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:56:05,881 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 in testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:05,881 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/9ec888a27b674bbcbeb22fc8bfc3689b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/99f4f88d3868413ebcf823d091ee7705, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/89b7b91027a4478089bd339b4d4bc1f7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/d9c69fd270774e49a0408b7c9da88356, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/654750f0de9d494cbac9afcb59e40fb7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/c81c5181a96542bdba278eb6c995f674, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/6a8a63a4e22a4046be24bdee6cb8026a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/5c14574da15e40a789e02dc0cef95c7a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/a5754e5742c24947b7871e2969eaec80, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/42bac1fb639a4f628a31bf6f40cecd73_SeqId_44_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp, totalSize=49.7 K 2023-02-09 09:56:05,881 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 9ec888a27b674bbcbeb22fc8bfc3689b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 09:56:05,881 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 99f4f88d3868413ebcf823d091ee7705, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 09:56:05,881 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 89b7b91027a4478089bd339b4d4bc1f7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 09:56:05,881 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting d9c69fd270774e49a0408b7c9da88356, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 09:56:05,881 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 654750f0de9d494cbac9afcb59e40fb7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 09:56:05,881 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting c81c5181a96542bdba278eb6c995f674, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 09:56:05,881 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 6a8a63a4e22a4046be24bdee6cb8026a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 09:56:05,881 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 5c14574da15e40a789e02dc0cef95c7a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 09:56:05,881 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting a5754e5742c24947b7871e2969eaec80, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 09:56:05,881 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 42bac1fb639a4f628a31bf6f40cecd73_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 09:56:05,883 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:56:05,884 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:05,885 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601017b0126274f1c46dd89291e8657554e26 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601017b0126274f1c46dd89291e8657554e26 2023-02-09 09:56:05,886 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010301e14f162cb44574b620c0d5c3599554 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010301e14f162cb44574b620c0d5c3599554 2023-02-09 09:56:05,887 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-09 09:56:05,887 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-09 09:56:05,887 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:56:05,887 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:56:05,908 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2e714ed6a371d3ab3e132b9b4cdecb8f#family1#compaction#165 average throughput is 0.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-09 09:56:06,332 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ccccc1303a4e44daa2229a5dd6eb3899 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231ccccc1303a4e44daa2229a5dd6eb3899 2023-02-09 09:56:06,333 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:56:06,355 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:06,356 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:06,363 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d2fba83b886e40ae8f9e87dc483ecd26 first=Optional[r1] last=Optional[r4] 2023-02-09 09:56:06,364 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d2fba83b886e40ae8f9e87dc483ecd26}] 2023-02-09 09:56:06,366 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d2fba83b886e40ae8f9e87dc483ecd26 for inclusion in 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 2023-02-09 09:56:06,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-09 09:56:06,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:56:06,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:06,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d2fba83b886e40ae8f9e87dc483ecd26 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__m6fvkuclfmut8830u88b004fkdrlv9ssf5n234rntub2akoo2g2f0qi3ptslhhka/family1/d2fba83b886e40ae8f9e87dc483ecd26 2023-02-09 09:56:06,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__m6fvkuclfmut8830u88b004fkdrlv9ssf5n234rntub2akoo2g2f0qi3ptslhhka/family1/d2fba83b886e40ae8f9e87dc483ecd26 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/94ef273f50494faeb6706b3515f042d9_SeqId_46_ 2023-02-09 09:56:06,371 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__m6fvkuclfmut8830u88b004fkdrlv9ssf5n234rntub2akoo2g2f0qi3ptslhhka/family1/d2fba83b886e40ae8f9e87dc483ecd26 into 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/94ef273f50494faeb6706b3515f042d9_SeqId_46_ - updating store file list. 2023-02-09 09:56:06,375 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/94ef273f50494faeb6706b3515f042d9_SeqId_46_ into 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 2023-02-09 09:56:06,375 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__m6fvkuclfmut8830u88b004fkdrlv9ssf5n234rntub2akoo2g2f0qi3ptslhhka/family1/d2fba83b886e40ae8f9e87dc483ecd26 into 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/94ef273f50494faeb6706b3515f042d9_SeqId_46_) 2023-02-09 09:56:06,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__m6fvkuclfmut8830u88b004fkdrlv9ssf5n234rntub2akoo2g2f0qi3ptslhhka/family1/d2fba83b886e40ae8f9e87dc483ecd26 2023-02-09 09:56:06,377 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:56:06,378 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:06,379 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201460f6b634ab04163bb2cb374577ead96 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201460f6b634ab04163bb2cb374577ead96 2023-02-09 09:56:06,380 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512057368e036084840699eea530672310cdd to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512057368e036084840699eea530672310cdd 2023-02-09 09:56:06,381 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228903cb30a18be45f3a6ceece4d5b0c8d7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228903cb30a18be45f3a6ceece4d5b0c8d7 2023-02-09 09:56:06,382 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123107cb787d7bb8423fbb4743e5a199b896 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123107cb787d7bb8423fbb4743e5a199b896 2023-02-09 09:56:06,382 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-09 09:56:06,382 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-09 09:56:06,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:56:06,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:56:06,727 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/10aa2604ecbd49f4b5ef8df6ce3d016b as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/10aa2604ecbd49f4b5ef8df6ce3d016b 2023-02-09 09:56:06,731 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 of 2e714ed6a371d3ab3e132b9b4cdecb8f into 10aa2604ecbd49f4b5ef8df6ce3d016b(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:56:06,731 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:06,732 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1675936565881; duration=0sec 2023-02-09 09:56:06,732 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:56:06,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/9ec888a27b674bbcbeb22fc8bfc3689b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/99f4f88d3868413ebcf823d091ee7705, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/89b7b91027a4478089bd339b4d4bc1f7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/d9c69fd270774e49a0408b7c9da88356, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/654750f0de9d494cbac9afcb59e40fb7, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/c81c5181a96542bdba278eb6c995f674, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/6a8a63a4e22a4046be24bdee6cb8026a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/5c14574da15e40a789e02dc0cef95c7a, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/a5754e5742c24947b7871e2969eaec80, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/42bac1fb639a4f628a31bf6f40cecd73_SeqId_44_] to archive 2023-02-09 09:56:06,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:06,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/9ec888a27b674bbcbeb22fc8bfc3689b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/9ec888a27b674bbcbeb22fc8bfc3689b 2023-02-09 09:56:06,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/99f4f88d3868413ebcf823d091ee7705 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/99f4f88d3868413ebcf823d091ee7705 2023-02-09 09:56:06,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/89b7b91027a4478089bd339b4d4bc1f7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/89b7b91027a4478089bd339b4d4bc1f7 2023-02-09 09:56:06,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/d9c69fd270774e49a0408b7c9da88356 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/d9c69fd270774e49a0408b7c9da88356 2023-02-09 09:56:06,782 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/654750f0de9d494cbac9afcb59e40fb7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/654750f0de9d494cbac9afcb59e40fb7 2023-02-09 09:56:06,783 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/c81c5181a96542bdba278eb6c995f674 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/c81c5181a96542bdba278eb6c995f674 2023-02-09 09:56:06,784 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/6a8a63a4e22a4046be24bdee6cb8026a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/6a8a63a4e22a4046be24bdee6cb8026a 2023-02-09 09:56:06,785 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/5c14574da15e40a789e02dc0cef95c7a to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/5c14574da15e40a789e02dc0cef95c7a 2023-02-09 09:56:06,786 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/a5754e5742c24947b7871e2969eaec80 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/a5754e5742c24947b7871e2969eaec80 2023-02-09 09:56:06,787 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/42bac1fb639a4f628a31bf6f40cecd73_SeqId_44_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/42bac1fb639a4f628a31bf6f40cecd73_SeqId_44_ 2023-02-09 09:56:07,227 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113080811928250843f7bf8811428ba9f412 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113080811928250843f7bf8811428ba9f412 2023-02-09 09:56:07,228 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:56:07,243 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:07,244 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:07,250 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6ffa7039ba08464497a687bef8131231 first=Optional[r00] last=Optional[r8] 2023-02-09 09:56:07,251 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6ffa7039ba08464497a687bef8131231}] 2023-02-09 09:56:07,252 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6ffa7039ba08464497a687bef8131231 for inclusion in 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 2023-02-09 09:56:07,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-09 09:56:07,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 09:56:07,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:07,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6ffa7039ba08464497a687bef8131231 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__irhbd5ndjsr47k5l9d65a09o7pcclusuh1982lqa67cmk1k9ka1uhaph9l8vvg44/family1/6ffa7039ba08464497a687bef8131231 2023-02-09 09:56:07,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__irhbd5ndjsr47k5l9d65a09o7pcclusuh1982lqa67cmk1k9ka1uhaph9l8vvg44/family1/6ffa7039ba08464497a687bef8131231 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/4a0833b301e746abb3f1631f4ff3aac9_SeqId_49_ 2023-02-09 09:56:07,258 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__irhbd5ndjsr47k5l9d65a09o7pcclusuh1982lqa67cmk1k9ka1uhaph9l8vvg44/family1/6ffa7039ba08464497a687bef8131231 into 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/4a0833b301e746abb3f1631f4ff3aac9_SeqId_49_ - updating store file list. 2023-02-09 09:56:07,261 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/4a0833b301e746abb3f1631f4ff3aac9_SeqId_49_ into 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 2023-02-09 09:56:07,261 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__irhbd5ndjsr47k5l9d65a09o7pcclusuh1982lqa67cmk1k9ka1uhaph9l8vvg44/family1/6ffa7039ba08464497a687bef8131231 into 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/4a0833b301e746abb3f1631f4ff3aac9_SeqId_49_) 2023-02-09 09:56:07,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMinorCompactionWithMonthlyPolicy__irhbd5ndjsr47k5l9d65a09o7pcclusuh1982lqa67cmk1k9ka1uhaph9l8vvg44/family1/6ffa7039ba08464497a687bef8131231 2023-02-09 09:56:07,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 09:56:07,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 09:56:07,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(1912): 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 is initiating minor compaction (all files) 2023-02-09 09:56:07,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1a9956c4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:56:07,263 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 in testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:07,263 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/b523fc00d81d411080bdf1d88c96b492, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/10aa2604ecbd49f4b5ef8df6ce3d016b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/94ef273f50494faeb6706b3515f042d9_SeqId_46_, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/4a0833b301e746abb3f1631f4ff3aac9_SeqId_49_] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp, totalSize=344.3 K 2023-02-09 09:56:07,264 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting b523fc00d81d411080bdf1d88c96b492, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840054 2023-02-09 09:56:07,264 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 10aa2604ecbd49f4b5ef8df6ce3d016b, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240054 2023-02-09 09:56:07,264 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 94ef273f50494faeb6706b3515f042d9_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965240054 2023-02-09 09:56:07,265 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] compactions.Compactor(207): Compacting 4a0833b301e746abb3f1631f4ff3aac9_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014840054 2023-02-09 09:56:07,265 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:56:07,266 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:07,267 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120032d972a4b75408d8bc002a60b241b50 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120032d972a4b75408d8bc002a60b241b50 2023-02-09 09:56:07,268 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112873b115cc855e469ab8b0c5ad8d8d8752 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112873b115cc855e469ab8b0c5ad8d8d8752 2023-02-09 09:56:07,269 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130f4a4acc80ffa47019e143340b55f0e22 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130f4a4acc80ffa47019e143340b55f0e22 2023-02-09 09:56:07,269 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-09 09:56:07,269 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 09:56:07,270 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:56:07,270 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:56:07,270 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 09:56:07,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 09:56:07,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:56:07,272 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 09:56:07,274 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.2850 sec 2023-02-09 09:56:07,282 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2e714ed6a371d3ab3e132b9b4cdecb8f#family1#compaction#166 average throughput is 22.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-09 09:56:07,470 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=264 (was 251) - Thread LEAK? -, OpenFileDescriptor=657 (was 637) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=133 (was 112) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3456 (was 3837) 2023-02-09 09:56:07,485 INFO [Listener at localhost/42557] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=264, OpenFileDescriptor=657, MaxFileDescriptor=60000, SystemLoadAverage=133, ProcessCount=168, AvailableMemoryMB=3461 2023-02-09 09:56:07,515 DEBUG [Listener at localhost/42557] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-02-09 09:56:07,538 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:07,538 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:07,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04\xC9\xC2\x83\xBD{\x1D?|\x83\xD3\x8C\xA2\x94{\x0A"\x10\xB7!\x1Fi\x18\xFDy\xCC\xC8\xFFECeb\x86#*\x10\xF4gS\xF9A\xA3\x82\xC8]\x92_,U\xA8O(2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 09:56:07,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-02-09 09:56:07,573 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 09:56:07,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-02-09 09:56:07,573 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:07,573 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:07,573 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 09:56:07,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 09:56:07,575 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:07,575 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:07,575 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:07,576 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83 empty. 2023-02-09 09:56:07,576 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf empty. 2023-02-09 09:56:07,576 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e empty. 2023-02-09 09:56:07,576 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:07,576 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:07,576 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:07,576 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-02-09 09:56:07,674 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:07,674 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:07,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 09:56:07,875 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:07,876 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:07,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 09:56:07,983 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-02-09 09:56:07,985 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7671): creating {ENCODED => 18d34fcb29417f486e3561257b2790bf, NAME => 'testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04\xC9\xC2\x83\xBD{\x1D?|\x83\xD3\x8C\xA2\x94{\x0A"\x10\xB7!\x1Fi\x18\xFDy\xCC\xC8\xFFECeb\x86#*\x10\xF4gS\xF9A\xA3\x82\xC8]\x92_,U\xA8O(2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:56:07,985 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7671): creating {ENCODED => bde72ffb8ef90a4019a13c516f644d0e, NAME => 'testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04\xC9\xC2\x83\xBD{\x1D?|\x83\xD3\x8C\xA2\x94{\x0A"\x10\xB7!\x1Fi\x18\xFDy\xCC\xC8\xFFECeb\x86#*\x10\xF4gS\xF9A\xA3\x82\xC8]\x92_,U\xA8O(2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:56:07,985 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7671): creating {ENCODED => 335e253892ba08cb7cdfed22db4c2e83, NAME => 'testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04\xC9\xC2\x83\xBD{\x1D?|\x83\xD3\x8C\xA2\x94{\x0A"\x10\xB7!\x1Fi\x18\xFDy\xCC\xC8\xFFECeb\x86#*\x10\xF4gS\xF9A\xA3\x82\xC8]\x92_,U\xA8O(2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/.tmp 2023-02-09 09:56:07,999 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:56:07,999 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1603): Closing bde72ffb8ef90a4019a13c516f644d0e, disabling compactions & flushes 2023-02-09 09:56:07,999 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:07,999 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:07,999 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. after waiting 0 ms 2023-02-09 09:56:07,999 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:07,999 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:07,999 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1557): Region close journal for bde72ffb8ef90a4019a13c516f644d0e: Waiting for close lock at 1675936567999Disabling compacts and flushes for region at 1675936567999Disabling writes for close at 1675936567999Writing region close event to WAL at 1675936567999Closed at 1675936567999 2023-02-09 09:56:08,107 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/.tmp/family1/e1d3e12cdfc44bf5837eec01095ad759 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/e1d3e12cdfc44bf5837eec01095ad759 2023-02-09 09:56:08,112 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 2e714ed6a371d3ab3e132b9b4cdecb8f/family1 of 2e714ed6a371d3ab3e132b9b4cdecb8f into e1d3e12cdfc44bf5837eec01095ad759(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:56:08,113 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: 2023-02-09 09:56:08,113 INFO [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1675936567263; duration=0sec 2023-02-09 09:56:08,113 DEBUG [RS:0;jenkins-hbase4:42719-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 09:56:08,175 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/b523fc00d81d411080bdf1d88c96b492, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/10aa2604ecbd49f4b5ef8df6ce3d016b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/94ef273f50494faeb6706b3515f042d9_SeqId_46_, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/4a0833b301e746abb3f1631f4ff3aac9_SeqId_49_] to archive 2023-02-09 09:56:08,176 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:08,177 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:08,177 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:08,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 09:56:08,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/b523fc00d81d411080bdf1d88c96b492 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/b523fc00d81d411080bdf1d88c96b492 2023-02-09 09:56:08,179 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/10aa2604ecbd49f4b5ef8df6ce3d016b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/10aa2604ecbd49f4b5ef8df6ce3d016b 2023-02-09 09:56:08,180 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/94ef273f50494faeb6706b3515f042d9_SeqId_46_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/94ef273f50494faeb6706b3515f042d9_SeqId_46_ 2023-02-09 09:56:08,181 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/4a0833b301e746abb3f1631f4ff3aac9_SeqId_49_ to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/family1/4a0833b301e746abb3f1631f4ff3aac9_SeqId_49_ 2023-02-09 09:56:08,398 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:56:08,398 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1603): Closing 18d34fcb29417f486e3561257b2790bf, disabling compactions & flushes 2023-02-09 09:56:08,398 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:08,398 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:08,398 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. after waiting 0 ms 2023-02-09 09:56:08,398 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:08,398 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:08,398 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1557): Region close journal for 18d34fcb29417f486e3561257b2790bf: Waiting for close lock at 1675936568398Disabling compacts and flushes for region at 1675936568398Disabling writes for close at 1675936568398Writing region close event to WAL at 1675936568398Closed at 1675936568398 2023-02-09 09:56:08,398 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:56:08,398 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1603): Closing 335e253892ba08cb7cdfed22db4c2e83, disabling compactions & flushes 2023-02-09 09:56:08,398 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:08,399 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:08,399 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. after waiting 0 ms 2023-02-09 09:56:08,399 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:08,399 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:08,399 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1557): Region close journal for 335e253892ba08cb7cdfed22db4c2e83: Waiting for close lock at 1675936568398Disabling compacts and flushes for region at 1675936568398Disabling writes for close at 1675936568399 (+1 ms)Writing region close event to WAL at 1675936568399Closed at 1675936568399 2023-02-09 09:56:08,401 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 09:56:08,402 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675936568401"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936568401"}]},"ts":"1675936568401"} 2023-02-09 09:56:08,402 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936568401"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936568401"}]},"ts":"1675936568401"} 2023-02-09 09:56:08,402 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936568401"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675936568401"}]},"ts":"1675936568401"} 2023-02-09 09:56:08,403 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 09:56:08,404 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 09:56:08,404 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936568404"}]},"ts":"1675936568404"} 2023-02-09 09:56:08,405 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-02-09 09:56:08,409 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=335e253892ba08cb7cdfed22db4c2e83, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=bde72ffb8ef90a4019a13c516f644d0e, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=18d34fcb29417f486e3561257b2790bf, ASSIGN}] 2023-02-09 09:56:08,410 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=bde72ffb8ef90a4019a13c516f644d0e, ASSIGN 2023-02-09 09:56:08,410 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=335e253892ba08cb7cdfed22db4c2e83, ASSIGN 2023-02-09 09:56:08,411 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=18d34fcb29417f486e3561257b2790bf, ASSIGN 2023-02-09 09:56:08,411 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=bde72ffb8ef90a4019a13c516f644d0e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:56:08,411 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=18d34fcb29417f486e3561257b2790bf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:56:08,411 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=335e253892ba08cb7cdfed22db4c2e83, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42719,1675936416061; forceNewPlan=false, retain=false 2023-02-09 09:56:08,563 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=18d34fcb29417f486e3561257b2790bf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:08,563 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=bde72ffb8ef90a4019a13c516f644d0e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:08,563 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936568563"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936568563"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936568563"}]},"ts":"1675936568563"} 2023-02-09 09:56:08,563 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=335e253892ba08cb7cdfed22db4c2e83, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:08,563 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675936568563"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936568563"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936568563"}]},"ts":"1675936568563"} 2023-02-09 09:56:08,563 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936568563"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675936568563"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675936568563"}]},"ts":"1675936568563"} 2023-02-09 09:56:08,565 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE; OpenRegionProcedure 18d34fcb29417f486e3561257b2790bf, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:56:08,565 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=117, state=RUNNABLE; OpenRegionProcedure 335e253892ba08cb7cdfed22db4c2e83, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:56:08,566 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=118, state=RUNNABLE; OpenRegionProcedure bde72ffb8ef90a4019a13c516f644d0e, server=jenkins-hbase4.apache.org,42719,1675936416061}] 2023-02-09 09:56:08,678 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:08,678 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:08,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 09:56:08,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:08,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 18d34fcb29417f486e3561257b2790bf, NAME => 'testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 09:56:08,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:08,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:56:08,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:08,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:08,741 INFO [StoreOpener-18d34fcb29417f486e3561257b2790bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:08,743 DEBUG [StoreOpener-18d34fcb29417f486e3561257b2790bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1 2023-02-09 09:56:08,743 DEBUG [StoreOpener-18d34fcb29417f486e3561257b2790bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1 2023-02-09 09:56:08,743 INFO [StoreOpener-18d34fcb29417f486e3561257b2790bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18d34fcb29417f486e3561257b2790bf columnFamilyName family1 2023-02-09 09:56:08,744 INFO [StoreOpener-18d34fcb29417f486e3561257b2790bf-1] regionserver.HStore(310): Store=18d34fcb29417f486e3561257b2790bf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:56:08,744 INFO [StoreOpener-18d34fcb29417f486e3561257b2790bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:08,745 DEBUG [StoreOpener-18d34fcb29417f486e3561257b2790bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2 2023-02-09 09:56:08,745 DEBUG [StoreOpener-18d34fcb29417f486e3561257b2790bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2 2023-02-09 09:56:08,746 INFO [StoreOpener-18d34fcb29417f486e3561257b2790bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18d34fcb29417f486e3561257b2790bf columnFamilyName family2 2023-02-09 09:56:08,746 INFO [StoreOpener-18d34fcb29417f486e3561257b2790bf-1] regionserver.HStore(310): Store=18d34fcb29417f486e3561257b2790bf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:56:08,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:08,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:08,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:56:08,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:08,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:56:08,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 18d34fcb29417f486e3561257b2790bf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64195115, jitterRate=-0.043418243527412415}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:56:08,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 18d34fcb29417f486e3561257b2790bf: Running coprocessor pre-open hook at 1675936568720Writing region info on filesystem at 1675936568720Initializing all the Stores at 1675936568720Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04\xC9\xC2\x83\xBD{\x1D?|\x83\xD3\x8C\xA2\x94{\x0A"\x10\xB7!\x1Fi\x18\xFDy\xCC\xC8\xFFECeb\x86#*\x10\xF4gS\xF9A\xA3\x82\xC8]\x92_,U\xA8O(2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936568720Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936568721 (+1 ms)Cleaning up temporary data from old regions at 1675936568748 (+27 ms)Cleaning up detritus from prior splits at 1675936568748Running coprocessor post-open hooks at 1675936568752 (+4 ms)Region opened successfully at 1675936568752 2023-02-09 09:56:08,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf., pid=120, masterSystemTime=1675936568716 2023-02-09 09:56:08,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:08,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:08,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:08,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 335e253892ba08cb7cdfed22db4c2e83, NAME => 'testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 09:56:08,754 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=18d34fcb29417f486e3561257b2790bf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:08,755 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936568754"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936568754"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936568754"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936568754"}]},"ts":"1675936568754"} 2023-02-09 09:56:08,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:08,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:56:08,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:08,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:08,757 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-09 09:56:08,757 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; OpenRegionProcedure 18d34fcb29417f486e3561257b2790bf, server=jenkins-hbase4.apache.org,42719,1675936416061 in 191 msec 2023-02-09 09:56:08,758 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=18d34fcb29417f486e3561257b2790bf, ASSIGN in 348 msec 2023-02-09 09:56:08,776 INFO [StoreOpener-335e253892ba08cb7cdfed22db4c2e83-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:08,777 DEBUG [StoreOpener-335e253892ba08cb7cdfed22db4c2e83-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1 2023-02-09 09:56:08,777 DEBUG [StoreOpener-335e253892ba08cb7cdfed22db4c2e83-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1 2023-02-09 09:56:08,778 INFO [StoreOpener-335e253892ba08cb7cdfed22db4c2e83-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 335e253892ba08cb7cdfed22db4c2e83 columnFamilyName family1 2023-02-09 09:56:08,778 INFO [StoreOpener-335e253892ba08cb7cdfed22db4c2e83-1] regionserver.HStore(310): Store=335e253892ba08cb7cdfed22db4c2e83/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:56:08,779 INFO [StoreOpener-335e253892ba08cb7cdfed22db4c2e83-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:08,780 DEBUG [StoreOpener-335e253892ba08cb7cdfed22db4c2e83-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2 2023-02-09 09:56:08,780 DEBUG [StoreOpener-335e253892ba08cb7cdfed22db4c2e83-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2 2023-02-09 09:56:08,780 INFO [StoreOpener-335e253892ba08cb7cdfed22db4c2e83-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 335e253892ba08cb7cdfed22db4c2e83 columnFamilyName family2 2023-02-09 09:56:08,781 INFO [StoreOpener-335e253892ba08cb7cdfed22db4c2e83-1] regionserver.HStore(310): Store=335e253892ba08cb7cdfed22db4c2e83/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:56:08,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:08,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:08,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:56:08,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:08,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:56:08,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 335e253892ba08cb7cdfed22db4c2e83; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62260534, jitterRate=-0.07224574685096741}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:56:08,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 335e253892ba08cb7cdfed22db4c2e83: Running coprocessor pre-open hook at 1675936568755Writing region info on filesystem at 1675936568755Initializing all the Stores at 1675936568756 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04\xC9\xC2\x83\xBD{\x1D?|\x83\xD3\x8C\xA2\x94{\x0A"\x10\xB7!\x1Fi\x18\xFDy\xCC\xC8\xFFECeb\x86#*\x10\xF4gS\xF9A\xA3\x82\xC8]\x92_,U\xA8O(2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936568756Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936568756Cleaning up temporary data from old regions at 1675936568782 (+26 ms)Cleaning up detritus from prior splits at 1675936568782Running coprocessor post-open hooks at 1675936568786 (+4 ms)Region opened successfully at 1675936568786 2023-02-09 09:56:08,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83., pid=121, masterSystemTime=1675936568716 2023-02-09 09:56:08,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:08,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:08,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:08,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bde72ffb8ef90a4019a13c516f644d0e, NAME => 'testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 09:56:08,788 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=335e253892ba08cb7cdfed22db4c2e83, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:08,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:08,788 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675936568788"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936568788"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936568788"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936568788"}]},"ts":"1675936568788"} 2023-02-09 09:56:08,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 09:56:08,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:08,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:08,791 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=117 2023-02-09 09:56:08,791 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=117, state=SUCCESS; OpenRegionProcedure 335e253892ba08cb7cdfed22db4c2e83, server=jenkins-hbase4.apache.org,42719,1675936416061 in 224 msec 2023-02-09 09:56:08,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=335e253892ba08cb7cdfed22db4c2e83, ASSIGN in 382 msec 2023-02-09 09:56:08,809 INFO [StoreOpener-bde72ffb8ef90a4019a13c516f644d0e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:08,811 DEBUG [StoreOpener-bde72ffb8ef90a4019a13c516f644d0e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1 2023-02-09 09:56:08,811 DEBUG [StoreOpener-bde72ffb8ef90a4019a13c516f644d0e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1 2023-02-09 09:56:08,811 INFO [StoreOpener-bde72ffb8ef90a4019a13c516f644d0e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bde72ffb8ef90a4019a13c516f644d0e columnFamilyName family1 2023-02-09 09:56:08,812 INFO [StoreOpener-bde72ffb8ef90a4019a13c516f644d0e-1] regionserver.HStore(310): Store=bde72ffb8ef90a4019a13c516f644d0e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:56:08,812 INFO [StoreOpener-bde72ffb8ef90a4019a13c516f644d0e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:08,813 DEBUG [StoreOpener-bde72ffb8ef90a4019a13c516f644d0e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2 2023-02-09 09:56:08,813 DEBUG [StoreOpener-bde72ffb8ef90a4019a13c516f644d0e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2 2023-02-09 09:56:08,813 INFO [StoreOpener-bde72ffb8ef90a4019a13c516f644d0e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bde72ffb8ef90a4019a13c516f644d0e columnFamilyName family2 2023-02-09 09:56:08,813 INFO [StoreOpener-bde72ffb8ef90a4019a13c516f644d0e-1] regionserver.HStore(310): Store=bde72ffb8ef90a4019a13c516f644d0e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 09:56:08,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:08,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:08,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 09:56:08,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:08,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 09:56:08,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bde72ffb8ef90a4019a13c516f644d0e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58823872, jitterRate=-0.12345600128173828}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 09:56:08,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bde72ffb8ef90a4019a13c516f644d0e: Running coprocessor pre-open hook at 1675936568788Writing region info on filesystem at 1675936568788Initializing all the Stores at 1675936568789 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04\xC9\xC2\x83\xBD{\x1D?|\x83\xD3\x8C\xA2\x94{\x0A"\x10\xB7!\x1Fi\x18\xFDy\xCC\xC8\xFFECeb\x86#*\x10\xF4gS\xF9A\xA3\x82\xC8]\x92_,U\xA8O(2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936568789Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675936568789Cleaning up temporary data from old regions at 1675936568815 (+26 ms)Cleaning up detritus from prior splits at 1675936568815Running coprocessor post-open hooks at 1675936568818 (+3 ms)Region opened successfully at 1675936568818 2023-02-09 09:56:08,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e., pid=122, masterSystemTime=1675936568716 2023-02-09 09:56:08,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:08,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:08,820 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=bde72ffb8ef90a4019a13c516f644d0e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:08,820 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675936568820"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675936568820"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675936568820"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675936568820"}]},"ts":"1675936568820"} 2023-02-09 09:56:08,822 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=118 2023-02-09 09:56:08,822 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=118, state=SUCCESS; OpenRegionProcedure bde72ffb8ef90a4019a13c516f644d0e, server=jenkins-hbase4.apache.org,42719,1675936416061 in 255 msec 2023-02-09 09:56:08,823 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-09 09:56:08,823 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=bde72ffb8ef90a4019a13c516f644d0e, ASSIGN in 413 msec 2023-02-09 09:56:08,824 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 09:56:08,824 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675936568824"}]},"ts":"1675936568824"} 2023-02-09 09:56:08,825 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-02-09 09:56:08,827 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 09:56:08,827 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2570 sec 2023-02-09 09:56:09,679 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:09,679 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:09,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 09:56:09,680 INFO [Listener at localhost/42557] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-02-09 09:56:09,686 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:09,687 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:09,687 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:09,689 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:09,689 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:09,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:56:09,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 09:56:09,692 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 09:56:09,692 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:09,692 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 09:56:09,692 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 09:56:09,692 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:09,692 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:09,694 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:09,694 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:09,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:09,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:09,694 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:09,694 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:56:09,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:09,694 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:09,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:56:09,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:09,695 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 09:56:09,695 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 09:56:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 09:56:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 09:56:09,697 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:09,697 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. started... 2023-02-09 09:56:09,697 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:09,697 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. started... 2023-02-09 09:56:09,698 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bde72ffb8ef90a4019a13c516f644d0e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 09:56:09,697 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:56:09,699 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 18d34fcb29417f486e3561257b2790bf 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 09:56:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:56:09,698 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:09,699 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. started... 2023-02-09 09:56:09,700 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 335e253892ba08cb7cdfed22db4c2e83 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 09:56:09,760 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209694be9505cf74d7a93ec6a6f31a6a390 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209694be9505cf74d7a93ec6a6f31a6a390 2023-02-09 09:56:10,158 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302091eff1cc74a274f32898f1e72be9f2498 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302091eff1cc74a274f32898f1e72be9f2498 2023-02-09 09:56:10,158 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302093f09e40fb8d244028c08ed6a8e821a98 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302093f09e40fb8d244028c08ed6a8e821a98 2023-02-09 09:56:10,179 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/39222d0884d941349b013d516ea39c7c 2023-02-09 09:56:10,574 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/6e784ae3f16446f9809584e47435ee62 2023-02-09 09:56:10,574 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/b2b7a492acff4c14a622f5648887b434 2023-02-09 09:56:10,613 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302090a6b98e5e9044870b9c288c35b9fb995 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202302090a6b98e5e9044870b9c288c35b9fb995 2023-02-09 09:56:10,617 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/40ed25d338414fafbc061673e86018b3 2023-02-09 09:56:10,620 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/39222d0884d941349b013d516ea39c7c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/39222d0884d941349b013d516ea39c7c 2023-02-09 09:56:10,644 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/39222d0884d941349b013d516ea39c7c, entries=4, sequenceid=5, filesize=5.5 K 2023-02-09 09:56:10,645 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/40ed25d338414fafbc061673e86018b3 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/40ed25d338414fafbc061673e86018b3 2023-02-09 09:56:10,648 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/40ed25d338414fafbc061673e86018b3, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 09:56:10,648 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for bde72ffb8ef90a4019a13c516f644d0e in 950ms, sequenceid=5, compaction requested=false 2023-02-09 09:56:10,648 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-09 09:56:10,649 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bde72ffb8ef90a4019a13c516f644d0e: 2023-02-09 09:56:10,649 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:10,649 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:56:11,009 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302097dc3ca5de1d54107a49905ef02137dcb to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302097dc3ca5de1d54107a49905ef02137dcb 2023-02-09 09:56:11,010 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209035db02161f44e4e9c6116d0eaf1ca6e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230209035db02161f44e4e9c6116d0eaf1ca6e 2023-02-09 09:56:11,014 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/e76f31e597b743928ec44472dafae196 2023-02-09 09:56:11,018 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/6e784ae3f16446f9809584e47435ee62 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/6e784ae3f16446f9809584e47435ee62 2023-02-09 09:56:11,042 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/6e784ae3f16446f9809584e47435ee62, entries=4, sequenceid=5, filesize=5.5 K 2023-02-09 09:56:11,043 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/e76f31e597b743928ec44472dafae196 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/e76f31e597b743928ec44472dafae196 2023-02-09 09:56:11,046 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/e76f31e597b743928ec44472dafae196, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 09:56:11,047 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 18d34fcb29417f486e3561257b2790bf in 1348ms, sequenceid=5, compaction requested=false 2023-02-09 09:56:11,048 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 18d34fcb29417f486e3561257b2790bf: 2023-02-09 09:56:11,048 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:11,048 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:56:11,414 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/9aa3a2ca5de247eaa90ff8ce3cd3a92c 2023-02-09 09:56:11,418 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/b2b7a492acff4c14a622f5648887b434 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/b2b7a492acff4c14a622f5648887b434 2023-02-09 09:56:11,445 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/b2b7a492acff4c14a622f5648887b434, entries=4, sequenceid=5, filesize=5.5 K 2023-02-09 09:56:11,445 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/9aa3a2ca5de247eaa90ff8ce3cd3a92c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/9aa3a2ca5de247eaa90ff8ce3cd3a92c 2023-02-09 09:56:11,449 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/9aa3a2ca5de247eaa90ff8ce3cd3a92c, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 09:56:11,450 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 335e253892ba08cb7cdfed22db4c2e83 in 1750ms, sequenceid=5, compaction requested=false 2023-02-09 09:56:11,450 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 335e253892ba08cb7cdfed22db4c2e83: 2023-02-09 09:56:11,450 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:11,450 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:56:11,450 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:56:11,450 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,450 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 09:56:11,450 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 09:56:11,455 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,455 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:11,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:11,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:11,455 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:11,455 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:11,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:11,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:11,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:11,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:11,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 09:56:11,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@770193c[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:11,457 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 09:56:11,457 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:11,458 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:11,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:11,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:11,458 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 09:56:11,458 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 09:56:11,458 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:11,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,458 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:11,459 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,459 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 09:56:11,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:11,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:11,459 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:11,460 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 09:56:11,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:11,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:11,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:11,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:11,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:11,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:11,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 09:56:11,461 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 09:56:11,462 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:11,462 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:11,462 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 09:56:11,462 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:11,463 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:11,463 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:11,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:11,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:11,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:11,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:11,463 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:11,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:11,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:11,463 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:11,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:11,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:11,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:11,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:11,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:11,464 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,467 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,467 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:11,467 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:11,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:11,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 09:56:11,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:11,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:11,467 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:11,467 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:11,467 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:11,468 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,468 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:11,468 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:11,468 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:11,468 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:11,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:11,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:11,468 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:11,568 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:11,568 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:11,568 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:11,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:11,570 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:11,570 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:11,571 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:11,572 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:11,572 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:11,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:56:11,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 09:56:11,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 09:56:11,575 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 09:56:11,575 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:11,575 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 09:56:11,575 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 09:56:11,575 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:11,575 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:11,576 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,576 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:11,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:11,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:11,577 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:11,577 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:56:11,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:11,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:11,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:56:11,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:11,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 09:56:11,578 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 09:56:11,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 09:56:11,578 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 09:56:11,578 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:11,578 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 09:56:11,578 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:11,578 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:56:11,578 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:11,578 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:56:11,578 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. started... 2023-02-09 09:56:11,578 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:11,578 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. started... 2023-02-09 09:56:11,578 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. started... 2023-02-09 09:56:11,578 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 335e253892ba08cb7cdfed22db4c2e83 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 09:56:11,578 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bde72ffb8ef90a4019a13c516f644d0e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 09:56:11,578 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 18d34fcb29417f486e3561257b2790bf 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 09:56:11,622 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209015bdda5e4f845f293cab8835f9d5fab to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209015bdda5e4f845f293cab8835f9d5fab 2023-02-09 09:56:12,022 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209abad1f0b99cb4c97899a05a5055a37d7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209abad1f0b99cb4c97899a05a5055a37d7 2023-02-09 09:56:12,022 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020945001d7506f74c71929a9ad10268481c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023020945001d7506f74c71929a9ad10268481c 2023-02-09 09:56:12,037 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/d3e92d2d6a5e40c484f89cc752040885 2023-02-09 09:56:12,086 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020986e87ea7439144b6b6583d38558bfe92 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023020986e87ea7439144b6b6583d38558bfe92 2023-02-09 09:56:12,091 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/7f05dd55c62144b9941f78e814192da0 2023-02-09 09:56:12,095 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/d3e92d2d6a5e40c484f89cc752040885 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/d3e92d2d6a5e40c484f89cc752040885 2023-02-09 09:56:12,120 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/d3e92d2d6a5e40c484f89cc752040885, entries=4, sequenceid=9, filesize=5.5 K 2023-02-09 09:56:12,121 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/7f05dd55c62144b9941f78e814192da0 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/7f05dd55c62144b9941f78e814192da0 2023-02-09 09:56:12,125 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/7f05dd55c62144b9941f78e814192da0, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 09:56:12,126 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 18d34fcb29417f486e3561257b2790bf in 548ms, sequenceid=9, compaction requested=false 2023-02-09 09:56:12,126 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 18d34fcb29417f486e3561257b2790bf: 2023-02-09 09:56:12,126 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:12,126 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:56:12,441 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/d1a4d889649e4992aaf5242c39198d0e 2023-02-09 09:56:12,444 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/545d8b7181914b2f9ff208d511a6a130 2023-02-09 09:56:12,880 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209b4a6edc64d6a4a1da1dd9aa977e70a55 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230209b4a6edc64d6a4a1da1dd9aa977e70a55 2023-02-09 09:56:12,886 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020918aa3a5fd4e74050b76eaf74fb8eb870 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023020918aa3a5fd4e74050b76eaf74fb8eb870 2023-02-09 09:56:13,284 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/aea9e1ae09114fe580fd7901872f3a57 2023-02-09 09:56:13,288 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/d1a4d889649e4992aaf5242c39198d0e as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/d1a4d889649e4992aaf5242c39198d0e 2023-02-09 09:56:13,291 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/ede23a37f59d415ab61021f356ca4103 2023-02-09 09:56:13,294 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/545d8b7181914b2f9ff208d511a6a130 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/545d8b7181914b2f9ff208d511a6a130 2023-02-09 09:56:13,313 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/d1a4d889649e4992aaf5242c39198d0e, entries=4, sequenceid=9, filesize=5.5 K 2023-02-09 09:56:13,313 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/aea9e1ae09114fe580fd7901872f3a57 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/aea9e1ae09114fe580fd7901872f3a57 2023-02-09 09:56:13,317 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/aea9e1ae09114fe580fd7901872f3a57, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 09:56:13,317 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 335e253892ba08cb7cdfed22db4c2e83 in 1739ms, sequenceid=9, compaction requested=false 2023-02-09 09:56:13,318 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 335e253892ba08cb7cdfed22db4c2e83: 2023-02-09 09:56:13,318 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:13,318 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:56:13,329 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/545d8b7181914b2f9ff208d511a6a130, entries=4, sequenceid=9, filesize=5.5 K 2023-02-09 09:56:13,330 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/ede23a37f59d415ab61021f356ca4103 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/ede23a37f59d415ab61021f356ca4103 2023-02-09 09:56:13,333 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/ede23a37f59d415ab61021f356ca4103, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 09:56:13,334 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for bde72ffb8ef90a4019a13c516f644d0e in 1756ms, sequenceid=9, compaction requested=false 2023-02-09 09:56:13,334 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bde72ffb8ef90a4019a13c516f644d0e: 2023-02-09 09:56:13,334 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:13,334 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:56:13,335 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:56:13,335 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,335 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 09:56:13,335 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 09:56:13,336 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,336 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:13,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:13,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:13,337 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:13,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:13,337 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:13,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:13,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:13,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:13,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 09:56:13,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2160dcdf[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:13,338 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 09:56:13,338 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:13,339 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:13,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:13,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:13,339 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 09:56:13,339 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 09:56:13,339 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:13,340 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,340 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:13,347 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,347 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 09:56:13,347 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:13,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,347 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 09:56:13,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:13,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:13,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:13,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:13,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:13,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:13,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:13,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:13,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 09:56:13,349 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 09:56:13,349 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:13,349 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:13,349 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 09:56:13,349 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:13,351 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:13,351 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:13,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:13,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:13,351 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:13,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:13,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:13,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:13,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:13,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:13,352 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:13,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:13,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:13,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:13,353 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:13,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:13,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:13,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:13,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:13,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 09:56:13,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,355 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:13,355 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:13,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:13,355 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:13,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:13,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:13,456 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:13,456 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:13,456 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:13,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:13,458 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:13,458 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:13,458 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:13,460 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:13,460 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:13,460 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:56:13,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 09:56:13,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 09:56:13,463 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 09:56:13,463 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:13,463 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 09:56:13,463 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 09:56:13,463 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:13,463 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:13,464 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:13,464 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:13,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:13,465 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:13,465 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:56:13,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:13,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:13,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:56:13,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:13,465 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 09:56:13,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 09:56:13,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 09:56:13,466 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 09:56:13,466 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:13,466 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 09:56:13,466 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:13,466 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. started... 2023-02-09 09:56:13,466 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:13,466 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:56:13,466 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:13,466 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bde72ffb8ef90a4019a13c516f644d0e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 09:56:13,466 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. started... 2023-02-09 09:56:13,466 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:56:13,466 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. started... 2023-02-09 09:56:13,466 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 335e253892ba08cb7cdfed22db4c2e83 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 09:56:13,466 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 18d34fcb29417f486e3561257b2790bf 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 09:56:13,910 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209430a7bf7f682448fa14f9565d1afb7e0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209430a7bf7f682448fa14f9565d1afb7e0 2023-02-09 09:56:13,921 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209bd5cc0005f9244c2ab3c5e3d00b71b4c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209bd5cc0005f9244c2ab3c5e3d00b71b4c 2023-02-09 09:56:13,922 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209d1d8495117e041e0bb9019e6820f7b82 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209d1d8495117e041e0bb9019e6820f7b82 2023-02-09 09:56:13,946 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:56:13,946 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x02465de9 to 127.0.0.1:64170 2023-02-09 09:56:13,946 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:56:13,947 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:56:13,947 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2728fdbe to 127.0.0.1:64170 2023-02-09 09:56:13,947 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:56:14,271 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 09:56:14,353 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/b42d68cf547745eab54326934f155089 2023-02-09 09:56:14,357 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/422c7b6e7cc64a9182706d62560708cf 2023-02-09 09:56:14,357 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/da87d6049eb847d4af9de9559a36ce01 2023-02-09 09:56:14,414 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302093d9b301343b8448f8ebedcd340f2b362 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302093d9b301343b8448f8ebedcd340f2b362 2023-02-09 09:56:14,421 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209f9059cbfc8964c9286ff690792bbd2a7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230209f9059cbfc8964c9286ff690792bbd2a7 2023-02-09 09:56:14,428 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/b3b2446deeab491eae2c9046b348d702 2023-02-09 09:56:14,433 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/da87d6049eb847d4af9de9559a36ce01 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/da87d6049eb847d4af9de9559a36ce01 2023-02-09 09:56:14,437 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/32f486db13ca40fa830c1282061519b8 2023-02-09 09:56:14,442 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/422c7b6e7cc64a9182706d62560708cf as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/422c7b6e7cc64a9182706d62560708cf 2023-02-09 09:56:14,464 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/422c7b6e7cc64a9182706d62560708cf, entries=4, sequenceid=13, filesize=5.5 K 2023-02-09 09:56:14,465 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/32f486db13ca40fa830c1282061519b8 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/32f486db13ca40fa830c1282061519b8 2023-02-09 09:56:14,466 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/da87d6049eb847d4af9de9559a36ce01, entries=4, sequenceid=13, filesize=5.5 K 2023-02-09 09:56:14,467 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/b3b2446deeab491eae2c9046b348d702 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/b3b2446deeab491eae2c9046b348d702 2023-02-09 09:56:14,469 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/32f486db13ca40fa830c1282061519b8, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 09:56:14,470 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 18d34fcb29417f486e3561257b2790bf in 1004ms, sequenceid=13, compaction requested=true 2023-02-09 09:56:14,470 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 18d34fcb29417f486e3561257b2790bf: 2023-02-09 09:56:14,470 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:14,470 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:56:14,471 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/b3b2446deeab491eae2c9046b348d702, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 09:56:14,471 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 335e253892ba08cb7cdfed22db4c2e83 in 1005ms, sequenceid=13, compaction requested=true 2023-02-09 09:56:14,471 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 335e253892ba08cb7cdfed22db4c2e83: 2023-02-09 09:56:14,471 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:14,471 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:56:14,815 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209d84f46e8ec3742b4bc36790df359ce42 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230209d84f46e8ec3742b4bc36790df359ce42 2023-02-09 09:56:15,220 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/58f046e8343e4050ad6d46ca19e91086 2023-02-09 09:56:15,224 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/b42d68cf547745eab54326934f155089 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/b42d68cf547745eab54326934f155089 2023-02-09 09:56:15,247 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/b42d68cf547745eab54326934f155089, entries=4, sequenceid=13, filesize=5.5 K 2023-02-09 09:56:15,247 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/58f046e8343e4050ad6d46ca19e91086 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/58f046e8343e4050ad6d46ca19e91086 2023-02-09 09:56:15,250 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/58f046e8343e4050ad6d46ca19e91086, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 09:56:15,251 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for bde72ffb8ef90a4019a13c516f644d0e in 1785ms, sequenceid=13, compaction requested=true 2023-02-09 09:56:15,251 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bde72ffb8ef90a4019a13c516f644d0e: 2023-02-09 09:56:15,251 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:15,251 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:56:15,251 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:56:15,251 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,251 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 09:56:15,251 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 09:56:15,253 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,253 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:15,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:15,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:15,254 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:15,254 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:15,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:15,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:15,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 09:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77b58ae9[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:15,255 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 09:56:15,255 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:15,256 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:15,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:15,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:15,256 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 09:56:15,256 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 09:56:15,256 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:15,256 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,256 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:15,257 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,257 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 09:56:15,257 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:15,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,258 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 09:56:15,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:15,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:15,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:15,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:15,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:15,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:15,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:15,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:15,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 09:56:15,259 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 09:56:15,259 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:15,259 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:15,260 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 09:56:15,260 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:15,261 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:15,261 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:15,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:15,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:15,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:15,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:15,261 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:15,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:15,262 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:15,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:15,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:15,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:15,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:15,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:15,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:15,263 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:15,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:15,266 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:15,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:15,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:15,266 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 09:56:15,266 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:15,266 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:15,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:15,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:15,266 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:15,367 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:15,367 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:15,367 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:15,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:15,368 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:15,369 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:15,369 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 09:56:15,370 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:15,370 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:15,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:56:15,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 09:56:15,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 09:56:15,373 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 09:56:15,373 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:15,373 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 09:56:15,373 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 09:56:15,373 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:15,373 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:15,377 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:15,377 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:15,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:15,377 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:15,377 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:56:15,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:15,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:15,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:56:15,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:15,378 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 09:56:15,378 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 09:56:15,378 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 09:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 09:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 09:56:15,378 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:15,378 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. started... 2023-02-09 09:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:56:15,378 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:15,378 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:15,378 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. started... 2023-02-09 09:56:15,378 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bde72ffb8ef90a4019a13c516f644d0e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 09:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:56:15,378 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 335e253892ba08cb7cdfed22db4c2e83 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 09:56:15,378 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. started... 2023-02-09 09:56:15,379 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 18d34fcb29417f486e3561257b2790bf 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 09:56:15,421 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020999d63ccec08c47e7885f1e2347733810 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020999d63ccec08c47e7885f1e2347733810 2023-02-09 09:56:15,422 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302094f0126e944f6407d9228e96fdf55bd35 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302094f0126e944f6407d9228e96fdf55bd35 2023-02-09 09:56:15,436 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/a66ab766365045079458a2fa9ee1a136 2023-02-09 09:56:15,436 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209371deafa80c24c948273872aec766134 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209371deafa80c24c948273872aec766134 2023-02-09 09:56:15,437 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/5ff44b2a33d348559556cfd087da8bfa 2023-02-09 09:56:15,456 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/325ca484266d44b5b82f0137500d140d 2023-02-09 09:56:15,474 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020995c16926cb6a4b7f9292b107365d16f8 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023020995c16926cb6a4b7f9292b107365d16f8 2023-02-09 09:56:15,478 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/2d2f154d0b2042e4a49ca475cb8d9e9b 2023-02-09 09:56:15,482 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/a66ab766365045079458a2fa9ee1a136 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/a66ab766365045079458a2fa9ee1a136 2023-02-09 09:56:15,489 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302091afef68207894b1ba5413d3be3cb62e8 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302091afef68207894b1ba5413d3be3cb62e8 2023-02-09 09:56:15,507 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/a66ab766365045079458a2fa9ee1a136, entries=4, sequenceid=17, filesize=5.5 K 2023-02-09 09:56:15,508 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/2d2f154d0b2042e4a49ca475cb8d9e9b as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/2d2f154d0b2042e4a49ca475cb8d9e9b 2023-02-09 09:56:15,512 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/2d2f154d0b2042e4a49ca475cb8d9e9b, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 09:56:15,513 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 18d34fcb29417f486e3561257b2790bf in 134ms, sequenceid=17, compaction requested=true 2023-02-09 09:56:15,513 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 18d34fcb29417f486e3561257b2790bf: 2023-02-09 09:56:15,513 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:15,513 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:56:15,879 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209fba387d8317f417da3f1191ea7eb58b5 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230209fba387d8317f417da3f1191ea7eb58b5 2023-02-09 09:56:15,893 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/1b5066d054f24446b2a5fdc1c45eca33 2023-02-09 09:56:15,898 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/325ca484266d44b5b82f0137500d140d as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/325ca484266d44b5b82f0137500d140d 2023-02-09 09:56:15,921 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/325ca484266d44b5b82f0137500d140d, entries=4, sequenceid=17, filesize=5.5 K 2023-02-09 09:56:15,922 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/1b5066d054f24446b2a5fdc1c45eca33 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/1b5066d054f24446b2a5fdc1c45eca33 2023-02-09 09:56:15,925 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/1b5066d054f24446b2a5fdc1c45eca33, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 09:56:15,926 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 335e253892ba08cb7cdfed22db4c2e83 in 548ms, sequenceid=17, compaction requested=true 2023-02-09 09:56:15,926 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 335e253892ba08cb7cdfed22db4c2e83: 2023-02-09 09:56:15,926 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:15,926 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:56:16,284 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/0d52aa8345a34e6386907f9960fd4133 2023-02-09 09:56:16,288 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/5ff44b2a33d348559556cfd087da8bfa as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/5ff44b2a33d348559556cfd087da8bfa 2023-02-09 09:56:16,311 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/5ff44b2a33d348559556cfd087da8bfa, entries=4, sequenceid=17, filesize=5.5 K 2023-02-09 09:56:16,311 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/0d52aa8345a34e6386907f9960fd4133 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/0d52aa8345a34e6386907f9960fd4133 2023-02-09 09:56:16,315 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/0d52aa8345a34e6386907f9960fd4133, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 09:56:16,316 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for bde72ffb8ef90a4019a13c516f644d0e in 938ms, sequenceid=17, compaction requested=true 2023-02-09 09:56:16,316 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bde72ffb8ef90a4019a13c516f644d0e: 2023-02-09 09:56:16,316 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:16,316 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:56:16,316 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:56:16,316 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,316 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 09:56:16,316 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 09:56:16,318 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:16,318 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:16,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:16,318 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:16,318 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:16,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:16,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:16,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:16,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:16,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 09:56:16,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64ffa425[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:16,319 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 09:56:16,319 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:16,321 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:16,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:16,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:16,321 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 09:56:16,321 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 09:56:16,321 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:16,321 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,321 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:16,322 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,322 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 09:56:16,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,322 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:16,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:16,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:16,323 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 09:56:16,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:16,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:16,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:16,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:16,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:16,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:16,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 09:56:16,325 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 09:56:16,325 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:16,325 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:16,325 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 09:56:16,325 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:16,326 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:16,326 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:16,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:16,326 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:16,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:16,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:16,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:16,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:16,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:16,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:16,326 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:16,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:16,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:16,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:16,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:16,327 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:16,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:16,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,329 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,329 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:16,329 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:16,329 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:16,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:16,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 09:56:16,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:16,329 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,330 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:16,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:16,329 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:16,330 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:16,330 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:16,330 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:16,330 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 09:56:16,330 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:16,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:16,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:16,330 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:16,430 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:16,431 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:16,431 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:16,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:16,439 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:56:16,445 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:56:16,452 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:56:16,453 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:56:16,540 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:16,541 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:16,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 09:56:16,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 09:56:16,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 09:56:16,543 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 09:56:16,543 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:16,543 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 09:56:16,543 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 09:56:16,543 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:16,543 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:16,546 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,546 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:16,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:16,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:16,546 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:16,546 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 09:56:16,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:16,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:16,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 09:56:16,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:16,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 09:56:16,547 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 09:56:16,547 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 09:56:16,547 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 09:56:16,547 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 09:56:16,547 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 09:56:16,547 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:16,547 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. started... 2023-02-09 09:56:16,547 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 09:56:16,548 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bde72ffb8ef90a4019a13c516f644d0e 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 09:56:16,548 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 09:56:16,548 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. started... 2023-02-09 09:56:16,548 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. started... 2023-02-09 09:56:16,548 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 18d34fcb29417f486e3561257b2790bf 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 09:56:16,548 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 335e253892ba08cb7cdfed22db4c2e83 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 09:56:16,987 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/2abcb660e3bf404088b618ebc61a4fe0 2023-02-09 09:56:16,996 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/a6fe8c4acfb14d1cb4bcff1cb617ea6b 2023-02-09 09:56:17,015 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2abcb660e3bf404088b618ebc61a4fe0 2023-02-09 09:56:17,024 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a6fe8c4acfb14d1cb4bcff1cb617ea6b 2023-02-09 09:56:17,388 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/895d812e350b41fb8d3330de01843348 2023-02-09 09:56:17,412 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 895d812e350b41fb8d3330de01843348 2023-02-09 09:56:17,430 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/1e82ab3dde354006a2f1b21de710a1ef 2023-02-09 09:56:17,433 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1e82ab3dde354006a2f1b21de710a1ef 2023-02-09 09:56:17,433 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/2abcb660e3bf404088b618ebc61a4fe0 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/2abcb660e3bf404088b618ebc61a4fe0 2023-02-09 09:56:17,435 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/a148c646498147439abfc5d875e1f9fe 2023-02-09 09:56:17,445 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a148c646498147439abfc5d875e1f9fe 2023-02-09 09:56:17,446 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/a6fe8c4acfb14d1cb4bcff1cb617ea6b as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/a6fe8c4acfb14d1cb4bcff1cb617ea6b 2023-02-09 09:56:17,459 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2abcb660e3bf404088b618ebc61a4fe0 2023-02-09 09:56:17,459 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/2abcb660e3bf404088b618ebc61a4fe0, entries=3, sequenceid=23, filesize=5.2 K 2023-02-09 09:56:17,459 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/1e82ab3dde354006a2f1b21de710a1ef as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/1e82ab3dde354006a2f1b21de710a1ef 2023-02-09 09:56:17,463 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1e82ab3dde354006a2f1b21de710a1ef 2023-02-09 09:56:17,463 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/1e82ab3dde354006a2f1b21de710a1ef, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 09:56:17,463 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 18d34fcb29417f486e3561257b2790bf in 915ms, sequenceid=23, compaction requested=true 2023-02-09 09:56:17,464 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 18d34fcb29417f486e3561257b2790bf: 2023-02-09 09:56:17,464 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:17,464 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 09:56:17,477 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a6fe8c4acfb14d1cb4bcff1cb617ea6b 2023-02-09 09:56:17,478 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/a6fe8c4acfb14d1cb4bcff1cb617ea6b, entries=3, sequenceid=23, filesize=5.2 K 2023-02-09 09:56:17,478 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/a148c646498147439abfc5d875e1f9fe as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/a148c646498147439abfc5d875e1f9fe 2023-02-09 09:56:17,482 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a148c646498147439abfc5d875e1f9fe 2023-02-09 09:56:17,482 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/a148c646498147439abfc5d875e1f9fe, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 09:56:17,482 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 335e253892ba08cb7cdfed22db4c2e83 in 934ms, sequenceid=23, compaction requested=true 2023-02-09 09:56:17,483 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 335e253892ba08cb7cdfed22db4c2e83: 2023-02-09 09:56:17,483 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:17,483 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 09:56:18,228 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/6cccdc69255443a8bee7e0b0a2b8be18 2023-02-09 09:56:18,231 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6cccdc69255443a8bee7e0b0a2b8be18 2023-02-09 09:56:18,232 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/895d812e350b41fb8d3330de01843348 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/895d812e350b41fb8d3330de01843348 2023-02-09 09:56:18,255 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 895d812e350b41fb8d3330de01843348 2023-02-09 09:56:18,255 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/895d812e350b41fb8d3330de01843348, entries=3, sequenceid=23, filesize=5.2 K 2023-02-09 09:56:18,255 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/6cccdc69255443a8bee7e0b0a2b8be18 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/6cccdc69255443a8bee7e0b0a2b8be18 2023-02-09 09:56:18,259 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6cccdc69255443a8bee7e0b0a2b8be18 2023-02-09 09:56:18,259 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/6cccdc69255443a8bee7e0b0a2b8be18, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 09:56:18,260 INFO [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for bde72ffb8ef90a4019a13c516f644d0e in 1712ms, sequenceid=23, compaction requested=true 2023-02-09 09:56:18,260 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bde72ffb8ef90a4019a13c516f644d0e: 2023-02-09 09:56:18,260 DEBUG [rs(jenkins-hbase4.apache.org,42719,1675936416061)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:18,260 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 09:56:18,260 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 09:56:18,260 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,260 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 09:56:18,260 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 09:56:18,261 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:18,261 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:18,262 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:18,262 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 09:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:18,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:18,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42719,1675936416061' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 09:56:18,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6341f87f[Count = 0] remaining members to acquire global barrier 2023-02-09 09:56:18,263 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 09:56:18,263 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:18,264 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:18,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:18,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:18,264 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 09:56:18,264 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 09:56:18,264 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42719,1675936416061' in zk 2023-02-09 09:56:18,264 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,264 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 09:56:18,265 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,265 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 09:56:18,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:18,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:18,265 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:18,265 DEBUG [member: 'jenkins-hbase4.apache.org,42719,1675936416061' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 09:56:18,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:18,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:18,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:18,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:18,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:18,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42719,1675936416061': 2023-02-09 09:56:18,267 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42719,1675936416061' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 09:56:18,267 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 09:56:18,267 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 09:56:18,267 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 09:56:18,267 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 09:56:18,267 INFO [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 09:56:18,270 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:18,270 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:18,270 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:18,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:18,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 09:56:18,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 09:56:18,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:18,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 09:56:18,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:18,271 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 09:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:18,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,272 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 09:56:18,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 09:56:18,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:18,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 09:56:18,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 09:56:18,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 09:56:18,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 09:56:18,274 DEBUG [(jenkins-hbase4.apache.org,33303,1675936414925)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:18,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:18,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 09:56:18,274 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 09:56:18,374 DEBUG [Listener at localhost/42557] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 09:56:18,375 INFO [Listener at localhost/42557] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:18,375 INFO [Listener at localhost/42557] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:18,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33303] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 09:56:18,376 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:56:18,376 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): bde72ffb8ef90a4019a13c516f644d0e/family1 is initiating major compaction (all files) 2023-02-09 09:56:18,377 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:56:18,377 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:56:18,379 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of bde72ffb8ef90a4019a13c516f644d0e/family1 in testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:18,379 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/39222d0884d941349b013d516ea39c7c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/545d8b7181914b2f9ff208d511a6a130, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/b42d68cf547745eab54326934f155089, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/5ff44b2a33d348559556cfd087da8bfa, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/895d812e350b41fb8d3330de01843348] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp, totalSize=27.1 K 2023-02-09 09:56:18,379 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 39222d0884d941349b013d516ea39c7c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936569687 2023-02-09 09:56:18,380 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 545d8b7181914b2f9ff208d511a6a130, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936571570 2023-02-09 09:56:18,380 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting b42d68cf547745eab54326934f155089, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936573458 2023-02-09 09:56:18,380 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 5ff44b2a33d348559556cfd087da8bfa, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936575369 2023-02-09 09:56:18,381 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 895d812e350b41fb8d3330de01843348, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:56:18,389 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): bde72ffb8ef90a4019a13c516f644d0e#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:56:18,834 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302096f9694cd6c134dd0ad6dbf566f96f286_del 2023-02-09 09:56:18,834 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302096f9694cd6c134dd0ad6dbf566f96f286_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302096f9694cd6c134dd0ad6dbf566f96f286_del 2023-02-09 09:56:19,285 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9de16ed6cf7f4019a0f5524b6a07fe19 2023-02-09 09:56:19,286 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family1/9de16ed6cf7f4019a0f5524b6a07fe19 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/9de16ed6cf7f4019a0f5524b6a07fe19 2023-02-09 09:56:19,308 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9de16ed6cf7f4019a0f5524b6a07fe19 2023-02-09 09:56:19,309 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bde72ffb8ef90a4019a13c516f644d0e/family1 of bde72ffb8ef90a4019a13c516f644d0e into 9de16ed6cf7f4019a0f5524b6a07fe19(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:56:19,309 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for bde72ffb8ef90a4019a13c516f644d0e: 2023-02-09 09:56:19,309 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:56:19,309 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): bde72ffb8ef90a4019a13c516f644d0e/family2 is initiating major compaction (all files) 2023-02-09 09:56:19,310 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:56:19,310 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:56:19,310 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of bde72ffb8ef90a4019a13c516f644d0e/family2 in testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:19,310 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/40ed25d338414fafbc061673e86018b3, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/ede23a37f59d415ab61021f356ca4103, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/58f046e8343e4050ad6d46ca19e91086, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/0d52aa8345a34e6386907f9960fd4133, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/6cccdc69255443a8bee7e0b0a2b8be18] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp, totalSize=25.4 K 2023-02-09 09:56:19,310 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 40ed25d338414fafbc061673e86018b3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936569687 2023-02-09 09:56:19,311 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting ede23a37f59d415ab61021f356ca4103, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936571570 2023-02-09 09:56:19,311 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 58f046e8343e4050ad6d46ca19e91086, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936573458 2023-02-09 09:56:19,311 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 0d52aa8345a34e6386907f9960fd4133, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936575369 2023-02-09 09:56:19,312 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 6cccdc69255443a8bee7e0b0a2b8be18, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:56:19,319 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): bde72ffb8ef90a4019a13c516f644d0e#family2#compaction#198 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:56:19,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/39222d0884d941349b013d516ea39c7c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/545d8b7181914b2f9ff208d511a6a130, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/b42d68cf547745eab54326934f155089, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/5ff44b2a33d348559556cfd087da8bfa, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/895d812e350b41fb8d3330de01843348] to archive 2023-02-09 09:56:19,376 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:19,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/39222d0884d941349b013d516ea39c7c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/39222d0884d941349b013d516ea39c7c 2023-02-09 09:56:19,378 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/545d8b7181914b2f9ff208d511a6a130 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/545d8b7181914b2f9ff208d511a6a130 2023-02-09 09:56:19,380 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/b42d68cf547745eab54326934f155089 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/b42d68cf547745eab54326934f155089 2023-02-09 09:56:19,380 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/5ff44b2a33d348559556cfd087da8bfa to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/5ff44b2a33d348559556cfd087da8bfa 2023-02-09 09:56:19,381 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/895d812e350b41fb8d3330de01843348 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/895d812e350b41fb8d3330de01843348 2023-02-09 09:56:19,727 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209467837df87e0427a88863428cf75e8c9_del 2023-02-09 09:56:19,727 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209467837df87e0427a88863428cf75e8c9_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230209467837df87e0427a88863428cf75e8c9_del 2023-02-09 09:56:20,539 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ade4275239164ed68bd5c30d70d905f8 2023-02-09 09:56:20,540 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/.tmp/family2/ade4275239164ed68bd5c30d70d905f8 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/ade4275239164ed68bd5c30d70d905f8 2023-02-09 09:56:20,543 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ade4275239164ed68bd5c30d70d905f8 2023-02-09 09:56:20,544 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bde72ffb8ef90a4019a13c516f644d0e/family2 of bde72ffb8ef90a4019a13c516f644d0e into ade4275239164ed68bd5c30d70d905f8(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:56:20,544 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for bde72ffb8ef90a4019a13c516f644d0e: 2023-02-09 09:56:20,544 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:56:20,544 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): 18d34fcb29417f486e3561257b2790bf/family1 is initiating major compaction (all files) 2023-02-09 09:56:20,545 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:56:20,545 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:56:20,545 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of 18d34fcb29417f486e3561257b2790bf/family1 in testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:20,545 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/6e784ae3f16446f9809584e47435ee62, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/d3e92d2d6a5e40c484f89cc752040885, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/422c7b6e7cc64a9182706d62560708cf, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/a66ab766365045079458a2fa9ee1a136, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/2abcb660e3bf404088b618ebc61a4fe0] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp, totalSize=27.1 K 2023-02-09 09:56:20,545 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 6e784ae3f16446f9809584e47435ee62, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936569687 2023-02-09 09:56:20,546 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting d3e92d2d6a5e40c484f89cc752040885, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936571571 2023-02-09 09:56:20,546 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 422c7b6e7cc64a9182706d62560708cf, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936573458 2023-02-09 09:56:20,546 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting a66ab766365045079458a2fa9ee1a136, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936575369 2023-02-09 09:56:20,547 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 2abcb660e3bf404088b618ebc61a4fe0, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:56:20,555 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): 18d34fcb29417f486e3561257b2790bf#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:56:20,576 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/40ed25d338414fafbc061673e86018b3, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/ede23a37f59d415ab61021f356ca4103, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/58f046e8343e4050ad6d46ca19e91086, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/0d52aa8345a34e6386907f9960fd4133, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/6cccdc69255443a8bee7e0b0a2b8be18] to archive 2023-02-09 09:56:20,577 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:20,578 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/40ed25d338414fafbc061673e86018b3 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/40ed25d338414fafbc061673e86018b3 2023-02-09 09:56:20,579 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/ede23a37f59d415ab61021f356ca4103 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/ede23a37f59d415ab61021f356ca4103 2023-02-09 09:56:20,580 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/58f046e8343e4050ad6d46ca19e91086 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/58f046e8343e4050ad6d46ca19e91086 2023-02-09 09:56:20,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/0d52aa8345a34e6386907f9960fd4133 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/0d52aa8345a34e6386907f9960fd4133 2023-02-09 09:56:20,582 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/6cccdc69255443a8bee7e0b0a2b8be18 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family2/6cccdc69255443a8bee7e0b0a2b8be18 2023-02-09 09:56:20,993 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209bcae2b28c4344be39b6e160bf9c1f752_del 2023-02-09 09:56:20,993 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209bcae2b28c4344be39b6e160bf9c1f752_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209bcae2b28c4344be39b6e160bf9c1f752_del 2023-02-09 09:56:21,861 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e51bb50d25344a0092a616b171ce65b2 2023-02-09 09:56:21,862 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family1/e51bb50d25344a0092a616b171ce65b2 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/e51bb50d25344a0092a616b171ce65b2 2023-02-09 09:56:21,884 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e51bb50d25344a0092a616b171ce65b2 2023-02-09 09:56:21,885 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 18d34fcb29417f486e3561257b2790bf/family1 of 18d34fcb29417f486e3561257b2790bf into e51bb50d25344a0092a616b171ce65b2(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:56:21,885 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for 18d34fcb29417f486e3561257b2790bf: 2023-02-09 09:56:21,885 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:56:21,885 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): 18d34fcb29417f486e3561257b2790bf/family2 is initiating major compaction (all files) 2023-02-09 09:56:21,885 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:56:21,885 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:56:21,886 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of 18d34fcb29417f486e3561257b2790bf/family2 in testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:21,886 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/e76f31e597b743928ec44472dafae196, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/7f05dd55c62144b9941f78e814192da0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/32f486db13ca40fa830c1282061519b8, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/2d2f154d0b2042e4a49ca475cb8d9e9b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/1e82ab3dde354006a2f1b21de710a1ef] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp, totalSize=25.4 K 2023-02-09 09:56:21,886 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting e76f31e597b743928ec44472dafae196, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936569687 2023-02-09 09:56:21,887 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 7f05dd55c62144b9941f78e814192da0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936571571 2023-02-09 09:56:21,887 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 32f486db13ca40fa830c1282061519b8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936573458 2023-02-09 09:56:21,887 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 2d2f154d0b2042e4a49ca475cb8d9e9b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936575369 2023-02-09 09:56:21,888 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 1e82ab3dde354006a2f1b21de710a1ef, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:56:21,895 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): 18d34fcb29417f486e3561257b2790bf#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:56:21,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/6e784ae3f16446f9809584e47435ee62, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/d3e92d2d6a5e40c484f89cc752040885, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/422c7b6e7cc64a9182706d62560708cf, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/a66ab766365045079458a2fa9ee1a136, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/2abcb660e3bf404088b618ebc61a4fe0] to archive 2023-02-09 09:56:21,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:21,977 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/6e784ae3f16446f9809584e47435ee62 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/6e784ae3f16446f9809584e47435ee62 2023-02-09 09:56:21,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/d3e92d2d6a5e40c484f89cc752040885 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/d3e92d2d6a5e40c484f89cc752040885 2023-02-09 09:56:21,979 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/422c7b6e7cc64a9182706d62560708cf to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/422c7b6e7cc64a9182706d62560708cf 2023-02-09 09:56:21,980 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/a66ab766365045079458a2fa9ee1a136 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/a66ab766365045079458a2fa9ee1a136 2023-02-09 09:56:21,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/2abcb660e3bf404088b618ebc61a4fe0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/2abcb660e3bf404088b618ebc61a4fe0 2023-02-09 09:56:22,302 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302097669e8f1486340b88161952a2ec38c50_del 2023-02-09 09:56:22,302 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302097669e8f1486340b88161952a2ec38c50_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302097669e8f1486340b88161952a2ec38c50_del 2023-02-09 09:56:22,714 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 66fdc0f14153418497e33a478ab4bdd1 2023-02-09 09:56:22,715 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/.tmp/family2/66fdc0f14153418497e33a478ab4bdd1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/66fdc0f14153418497e33a478ab4bdd1 2023-02-09 09:56:22,718 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 66fdc0f14153418497e33a478ab4bdd1 2023-02-09 09:56:22,719 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 18d34fcb29417f486e3561257b2790bf/family2 of 18d34fcb29417f486e3561257b2790bf into 66fdc0f14153418497e33a478ab4bdd1(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 09:56:22,719 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for 18d34fcb29417f486e3561257b2790bf: 2023-02-09 09:56:22,719 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:56:22,720 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): 335e253892ba08cb7cdfed22db4c2e83/family1 is initiating major compaction (all files) 2023-02-09 09:56:22,720 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:56:22,720 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:56:22,721 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of 335e253892ba08cb7cdfed22db4c2e83/family1 in testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:22,721 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/b2b7a492acff4c14a622f5648887b434, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/d1a4d889649e4992aaf5242c39198d0e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/da87d6049eb847d4af9de9559a36ce01, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/325ca484266d44b5b82f0137500d140d, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/a6fe8c4acfb14d1cb4bcff1cb617ea6b] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp, totalSize=27.1 K 2023-02-09 09:56:22,721 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting b2b7a492acff4c14a622f5648887b434, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936569686 2023-02-09 09:56:22,722 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting d1a4d889649e4992aaf5242c39198d0e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936571570 2023-02-09 09:56:22,722 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting da87d6049eb847d4af9de9559a36ce01, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936573458 2023-02-09 09:56:22,722 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 325ca484266d44b5b82f0137500d140d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936575368 2023-02-09 09:56:22,723 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting a6fe8c4acfb14d1cb4bcff1cb617ea6b, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:56:22,731 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): 335e253892ba08cb7cdfed22db4c2e83#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:56:22,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/e76f31e597b743928ec44472dafae196, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/7f05dd55c62144b9941f78e814192da0, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/32f486db13ca40fa830c1282061519b8, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/2d2f154d0b2042e4a49ca475cb8d9e9b, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/1e82ab3dde354006a2f1b21de710a1ef] to archive 2023-02-09 09:56:22,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:22,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/e76f31e597b743928ec44472dafae196 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/e76f31e597b743928ec44472dafae196 2023-02-09 09:56:22,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/7f05dd55c62144b9941f78e814192da0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/7f05dd55c62144b9941f78e814192da0 2023-02-09 09:56:22,782 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/32f486db13ca40fa830c1282061519b8 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/32f486db13ca40fa830c1282061519b8 2023-02-09 09:56:22,782 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/2d2f154d0b2042e4a49ca475cb8d9e9b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/2d2f154d0b2042e4a49ca475cb8d9e9b 2023-02-09 09:56:22,783 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/1e82ab3dde354006a2f1b21de710a1ef to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family2/1e82ab3dde354006a2f1b21de710a1ef 2023-02-09 09:56:23,169 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209f7b04d04b61440fa8e292f0e03e600d8_del 2023-02-09 09:56:23,169 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209f7b04d04b61440fa8e292f0e03e600d8_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209f7b04d04b61440fa8e292f0e03e600d8_del 2023-02-09 09:56:24,021 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 98d45c12a7234f6bbd95c15806ff95bb 2023-02-09 09:56:24,022 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family1/98d45c12a7234f6bbd95c15806ff95bb as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/98d45c12a7234f6bbd95c15806ff95bb 2023-02-09 09:56:24,044 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 98d45c12a7234f6bbd95c15806ff95bb 2023-02-09 09:56:24,045 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 335e253892ba08cb7cdfed22db4c2e83/family1 of 335e253892ba08cb7cdfed22db4c2e83 into 98d45c12a7234f6bbd95c15806ff95bb(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:56:24,045 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for 335e253892ba08cb7cdfed22db4c2e83: 2023-02-09 09:56:24,045 DEBUG [Listener at localhost/42557] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 09:56:24,045 DEBUG [Listener at localhost/42557] regionserver.HStore(1912): 335e253892ba08cb7cdfed22db4c2e83/family2 is initiating major compaction (all files) 2023-02-09 09:56:24,045 INFO [Listener at localhost/42557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 09:56:24,045 INFO [Listener at localhost/42557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 09:56:24,045 INFO [Listener at localhost/42557] regionserver.HRegion(2258): Starting compaction of 335e253892ba08cb7cdfed22db4c2e83/family2 in testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:24,045 INFO [Listener at localhost/42557] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/9aa3a2ca5de247eaa90ff8ce3cd3a92c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/aea9e1ae09114fe580fd7901872f3a57, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/b3b2446deeab491eae2c9046b348d702, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/1b5066d054f24446b2a5fdc1c45eca33, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/a148c646498147439abfc5d875e1f9fe] into tmpdir=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp, totalSize=25.4 K 2023-02-09 09:56:24,046 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 9aa3a2ca5de247eaa90ff8ce3cd3a92c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675936569686 2023-02-09 09:56:24,046 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting aea9e1ae09114fe580fd7901872f3a57, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675936571570 2023-02-09 09:56:24,046 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting b3b2446deeab491eae2c9046b348d702, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675936573458 2023-02-09 09:56:24,046 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting 1b5066d054f24446b2a5fdc1c45eca33, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675936575368 2023-02-09 09:56:24,047 DEBUG [Listener at localhost/42557] compactions.Compactor(207): Compacting a148c646498147439abfc5d875e1f9fe, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 09:56:24,054 INFO [Listener at localhost/42557] throttle.PressureAwareThroughputController(145): 335e253892ba08cb7cdfed22db4c2e83#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 09:56:24,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/b2b7a492acff4c14a622f5648887b434, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/d1a4d889649e4992aaf5242c39198d0e, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/da87d6049eb847d4af9de9559a36ce01, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/325ca484266d44b5b82f0137500d140d, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/a6fe8c4acfb14d1cb4bcff1cb617ea6b] to archive 2023-02-09 09:56:24,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:24,076 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/b2b7a492acff4c14a622f5648887b434 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/b2b7a492acff4c14a622f5648887b434 2023-02-09 09:56:24,077 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/d1a4d889649e4992aaf5242c39198d0e to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/d1a4d889649e4992aaf5242c39198d0e 2023-02-09 09:56:24,078 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/da87d6049eb847d4af9de9559a36ce01 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/da87d6049eb847d4af9de9559a36ce01 2023-02-09 09:56:24,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/325ca484266d44b5b82f0137500d140d to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/325ca484266d44b5b82f0137500d140d 2023-02-09 09:56:24,080 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/a6fe8c4acfb14d1cb4bcff1cb617ea6b to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/a6fe8c4acfb14d1cb4bcff1cb617ea6b 2023-02-09 09:56:24,462 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302097101d4813646485db24b63b569968537_del 2023-02-09 09:56:24,462 INFO [Listener at localhost/42557] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302097101d4813646485db24b63b569968537_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302097101d4813646485db24b63b569968537_del 2023-02-09 09:56:25,275 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03b2cc32204147808f75980b2231860c 2023-02-09 09:56:25,276 DEBUG [Listener at localhost/42557] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/.tmp/family2/03b2cc32204147808f75980b2231860c as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/03b2cc32204147808f75980b2231860c 2023-02-09 09:56:25,279 INFO [Listener at localhost/42557] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03b2cc32204147808f75980b2231860c 2023-02-09 09:56:25,280 INFO [Listener at localhost/42557] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 335e253892ba08cb7cdfed22db4c2e83/family2 of 335e253892ba08cb7cdfed22db4c2e83 into 03b2cc32204147808f75980b2231860c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 09:56:25,280 DEBUG [Listener at localhost/42557] regionserver.HRegion(2288): Compaction status journal for 335e253892ba08cb7cdfed22db4c2e83: 2023-02-09 09:56:25,283 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:56:25,286 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:56:25,289 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:56:25,289 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:56:25,294 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33303] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04\xC9\xC2\x83\xBD{\x1D?|\x83\xD3\x8C\xA2\x94{\x0A"\x10\xB7!\x1Fi\x18\xFDy\xCC\xC8\xFFECeb\x86#*\x10\xF4gS\xF9A\xA3\x82\xC8]\x92_,U\xA8O(2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-02-09 09:56:25,327 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-09 09:56:25,328 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-09 09:56:25,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:56:25,328 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675937185328 2023-02-09 09:56:25,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 09:56:25,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/9aa3a2ca5de247eaa90ff8ce3cd3a92c, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/aea9e1ae09114fe580fd7901872f3a57, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/b3b2446deeab491eae2c9046b348d702, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/1b5066d054f24446b2a5fdc1c45eca33, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/a148c646498147439abfc5d875e1f9fe] to archive 2023-02-09 09:56:25,376 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:25,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/9aa3a2ca5de247eaa90ff8ce3cd3a92c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/9aa3a2ca5de247eaa90ff8ce3cd3a92c 2023-02-09 09:56:25,379 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/aea9e1ae09114fe580fd7901872f3a57 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/aea9e1ae09114fe580fd7901872f3a57 2023-02-09 09:56:25,380 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/b3b2446deeab491eae2c9046b348d702 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/b3b2446deeab491eae2c9046b348d702 2023-02-09 09:56:25,381 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/1b5066d054f24446b2a5fdc1c45eca33 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/1b5066d054f24446b2a5fdc1c45eca33 2023-02-09 09:56:25,382 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/a148c646498147439abfc5d875e1f9fe to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family2/a148c646498147439abfc5d875e1f9fe 2023-02-09 09:56:25,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-02-09 09:56:25,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209f7b04d04b61440fa8e292f0e03e600d8_del 2023-02-09 09:56:25,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302096f9694cd6c134dd0ad6dbf566f96f286_del 2023-02-09 09:56:25,745 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209bcae2b28c4344be39b6e160bf9c1f752_del 2023-02-09 09:56:25,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 09:56:25,746 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] zookeeper.ReadOnlyZKClient(139): Connect 0x096775fe to 127.0.0.1:64170 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 09:56:25,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3b658a38, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 09:56:25,750 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 09:56:26,037 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 09:56:26,312 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302092de66256921d4316abd718cef1467e34 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302092de66256921d4316abd718cef1467e34 2023-02-09 09:56:26,313 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 09:56:26,324 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 09:56:26,326 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40536, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 09:56:26,332 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:26,332 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:26,332 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 09:56:26,334 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37850, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 09:56:26,363 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/7c58857083a04578a464282d82cc4d13 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-09 09:56:26,364 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/7c58857083a04578a464282d82cc4d13}] 2023-02-09 09:56:26,370 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/7c58857083a04578a464282d82cc4d13 for inclusion in 18d34fcb29417f486e3561257b2790bf/family1 2023-02-09 09:56:26,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-09 09:56:26,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 09:56:26,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 18d34fcb29417f486e3561257b2790bf: 2023-02-09 09:56:26,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/7c58857083a04578a464282d82cc4d13 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__oprmnvgk4d18rusmltaffi5t71qvo6jk2346qlc368ia1csq4pqjea9m7gjv8fv4/family1/7c58857083a04578a464282d82cc4d13 2023-02-09 09:56:26,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__oprmnvgk4d18rusmltaffi5t71qvo6jk2346qlc368ia1csq4pqjea9m7gjv8fv4/family1/7c58857083a04578a464282d82cc4d13 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/48ee2956df1843e6b8b78ec4e40f84a7_SeqId_28_ 2023-02-09 09:56:26,396 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__oprmnvgk4d18rusmltaffi5t71qvo6jk2346qlc368ia1csq4pqjea9m7gjv8fv4/family1/7c58857083a04578a464282d82cc4d13 into 18d34fcb29417f486e3561257b2790bf/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/48ee2956df1843e6b8b78ec4e40f84a7_SeqId_28_ - updating store file list. 2023-02-09 09:56:26,416 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/48ee2956df1843e6b8b78ec4e40f84a7_SeqId_28_ into 18d34fcb29417f486e3561257b2790bf/family1 2023-02-09 09:56:26,416 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__oprmnvgk4d18rusmltaffi5t71qvo6jk2346qlc368ia1csq4pqjea9m7gjv8fv4/family1/7c58857083a04578a464282d82cc4d13 into 18d34fcb29417f486e3561257b2790bf/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/family1/48ee2956df1843e6b8b78ec4e40f84a7_SeqId_28_) 2023-02-09 09:56:26,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__oprmnvgk4d18rusmltaffi5t71qvo6jk2346qlc368ia1csq4pqjea9m7gjv8fv4/family1/7c58857083a04578a464282d82cc4d13 2023-02-09 09:56:26,419 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 09:56:26,419 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:26,420 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209015bdda5e4f845f293cab8835f9d5fab to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209015bdda5e4f845f293cab8835f9d5fab 2023-02-09 09:56:26,421 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302093f09e40fb8d244028c08ed6a8e821a98 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302093f09e40fb8d244028c08ed6a8e821a98 2023-02-09 09:56:26,423 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020999d63ccec08c47e7885f1e2347733810 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020999d63ccec08c47e7885f1e2347733810 2023-02-09 09:56:26,424 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209bd5cc0005f9244c2ab3c5e3d00b71b4c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209bd5cc0005f9244c2ab3c5e3d00b71b4c 2023-02-09 09:56:26,424 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-09 09:56:26,424 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 09:56:26,424 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 09:56:26,424 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 09:56:27,275 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 09:56:27,381 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209b95fb3fa30624572a0f6d9ffc5586d7c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209b95fb3fa30624572a0f6d9ffc5586d7c 2023-02-09 09:56:27,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 09:56:27,396 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:27,396 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:27,424 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/dcbcd4e07a5b4e559da3d0084b647b05 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-09 09:56:27,425 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/dcbcd4e07a5b4e559da3d0084b647b05}] 2023-02-09 09:56:27,427 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/dcbcd4e07a5b4e559da3d0084b647b05 for inclusion in 335e253892ba08cb7cdfed22db4c2e83/family1 2023-02-09 09:56:27,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-09 09:56:27,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 09:56:27,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for 335e253892ba08cb7cdfed22db4c2e83: 2023-02-09 09:56:27,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/dcbcd4e07a5b4e559da3d0084b647b05 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__scr9gd4stvfqd49kdttdj4o0k76jtin2p7g6cs838hlcotktode9bfcn4a51c6rg/family1/dcbcd4e07a5b4e559da3d0084b647b05 2023-02-09 09:56:27,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__scr9gd4stvfqd49kdttdj4o0k76jtin2p7g6cs838hlcotktode9bfcn4a51c6rg/family1/dcbcd4e07a5b4e559da3d0084b647b05 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/dd6c8303d54a4df7880ff68f23c1ed74_SeqId_28_ 2023-02-09 09:56:27,463 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__scr9gd4stvfqd49kdttdj4o0k76jtin2p7g6cs838hlcotktode9bfcn4a51c6rg/family1/dcbcd4e07a5b4e559da3d0084b647b05 into 335e253892ba08cb7cdfed22db4c2e83/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/dd6c8303d54a4df7880ff68f23c1ed74_SeqId_28_ - updating store file list. 2023-02-09 09:56:27,484 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/dd6c8303d54a4df7880ff68f23c1ed74_SeqId_28_ into 335e253892ba08cb7cdfed22db4c2e83/family1 2023-02-09 09:56:27,484 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__scr9gd4stvfqd49kdttdj4o0k76jtin2p7g6cs838hlcotktode9bfcn4a51c6rg/family1/dcbcd4e07a5b4e559da3d0084b647b05 into 335e253892ba08cb7cdfed22db4c2e83/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/family1/dd6c8303d54a4df7880ff68f23c1ed74_SeqId_28_) 2023-02-09 09:56:27,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__scr9gd4stvfqd49kdttdj4o0k76jtin2p7g6cs838hlcotktode9bfcn4a51c6rg/family1/dcbcd4e07a5b4e559da3d0084b647b05 2023-02-09 09:56:27,486 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 09:56:27,487 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:27,488 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302091eff1cc74a274f32898f1e72be9f2498 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302091eff1cc74a274f32898f1e72be9f2498 2023-02-09 09:56:27,489 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209371deafa80c24c948273872aec766134 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209371deafa80c24c948273872aec766134 2023-02-09 09:56:27,490 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023020945001d7506f74c71929a9ad10268481c to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023020945001d7506f74c71929a9ad10268481c 2023-02-09 09:56:27,491 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209d1d8495117e041e0bb9019e6820f7b82 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209d1d8495117e041e0bb9019e6820f7b82 2023-02-09 09:56:27,491 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-09 09:56:27,491 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 09:56:27,491 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 09:56:27,491 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 09:56:28,118 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 09:56:28,466 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209f85a4f022570472c927f96b4f94b6a30 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209f85a4f022570472c927f96b4f94b6a30 2023-02-09 09:56:28,467 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 09:56:28,481 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 09:56:28,481 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 09:56:28,507 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/cb8bfe557500407abefff2b5e72dfa40 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-09 09:56:28,508 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e., hostname=jenkins-hbase4.apache.org,42719,1675936416061, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/cb8bfe557500407abefff2b5e72dfa40}] 2023-02-09 09:56:28,510 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(763): Validating hfile at hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/cb8bfe557500407abefff2b5e72dfa40 for inclusion in bde72ffb8ef90a4019a13c516f644d0e/family1 2023-02-09 09:56:28,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-09 09:56:28,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 09:56:28,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegion(2445): Flush status journal for bde72ffb8ef90a4019a13c516f644d0e: 2023-02-09 09:56:28,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/cb8bfe557500407abefff2b5e72dfa40 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__6uos0jr9m47ebpqqa1bkoerkioglvdo2o5qji8rrpiisskpeeovbn68sunqhgrab/family1/cb8bfe557500407abefff2b5e72dfa40 2023-02-09 09:56:28,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__6uos0jr9m47ebpqqa1bkoerkioglvdo2o5qji8rrpiisskpeeovbn68sunqhgrab/family1/cb8bfe557500407abefff2b5e72dfa40 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/fc04dfed95284eaa851b191824c22d90_SeqId_28_ 2023-02-09 09:56:28,534 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(849): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__6uos0jr9m47ebpqqa1bkoerkioglvdo2o5qji8rrpiisskpeeovbn68sunqhgrab/family1/cb8bfe557500407abefff2b5e72dfa40 into bde72ffb8ef90a4019a13c516f644d0e/family1 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/fc04dfed95284eaa851b191824c22d90_SeqId_28_ - updating store file list. 2023-02-09 09:56:28,556 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(882): Loaded HFile hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/fc04dfed95284eaa851b191824c22d90_SeqId_28_ into bde72ffb8ef90a4019a13c516f644d0e/family1 2023-02-09 09:56:28,556 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.HStore(855): Successfully loaded hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__6uos0jr9m47ebpqqa1bkoerkioglvdo2o5qji8rrpiisskpeeovbn68sunqhgrab/family1/cb8bfe557500407abefff2b5e72dfa40 into bde72ffb8ef90a4019a13c516f644d0e/family1 (new location: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/family1/fc04dfed95284eaa851b191824c22d90_SeqId_28_) 2023-02-09 09:56:28,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42719] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/staging/jenkins__testMajorCompactionFromAdmin__6uos0jr9m47ebpqqa1bkoerkioglvdo2o5qji8rrpiisskpeeovbn68sunqhgrab/family1/cb8bfe557500407abefff2b5e72dfa40 2023-02-09 09:56:28,558 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 09:56:28,558 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:28,559 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209430a7bf7f682448fa14f9565d1afb7e0 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209430a7bf7f682448fa14f9565d1afb7e0 2023-02-09 09:56:28,560 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302094f0126e944f6407d9228e96fdf55bd35 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302094f0126e944f6407d9228e96fdf55bd35 2023-02-09 09:56:28,561 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209694be9505cf74d7a93ec6a6f31a6a390 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209694be9505cf74d7a93ec6a6f31a6a390 2023-02-09 09:56:28,562 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209abad1f0b99cb4c97899a05a5055a37d7 to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209abad1f0b99cb4c97899a05a5055a37d7 2023-02-09 09:56:28,562 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-09 09:56:28,562 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 09:56:28,562 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 09:56:28,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 09:56:28,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 09:56:28,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209f7b04d04b61440fa8e292f0e03e600d8_del] 2023-02-09 09:56:28,563 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:28,564 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209f7b04d04b61440fa8e292f0e03e600d8_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209f7b04d04b61440fa8e292f0e03e600d8_del 2023-02-09 09:56:28,564 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-09 09:56:28,564 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302096f9694cd6c134dd0ad6dbf566f96f286_del] 2023-02-09 09:56:28,564 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:28,565 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302096f9694cd6c134dd0ad6dbf566f96f286_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302096f9694cd6c134dd0ad6dbf566f96f286_del 2023-02-09 09:56:28,565 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-09 09:56:28,565 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] compactions.PartitionedMobCompactor(367): [hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209bcae2b28c4344be39b6e160bf9c1f752_del] 2023-02-09 09:56:28,565 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 09:56:28,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209bcae2b28c4344be39b6e160bf9c1f752_del to hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209bcae2b28c4344be39b6e160bf9c1f752_del 2023-02-09 09:56:28,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-09 09:56:28,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675936419981] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 09:56:28,567 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-09 09:56:28,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.2420 sec 2023-02-09 09:56:28,582 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:56:28,584 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:56:28,586 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 09:56:28,587 DEBUG [Listener at localhost/42557] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 09:56:28,591 INFO [Listener at localhost/42557] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 09:56:28,627 INFO [Listener at localhost/42557] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=250 (was 264), OpenFileDescriptor=640 (was 657), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=152 (was 133) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3482 (was 3461) - AvailableMemoryMB LEAK? - 2023-02-09 09:56:28,627 DEBUG [Listener at localhost/42557] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6f46f74d to 127.0.0.1:64170 2023-02-09 09:56:28,628 DEBUG [Listener at localhost/42557] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:56:28,628 INFO [Listener at localhost/42557] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-09 09:56:28,628 INFO [Listener at localhost/42557] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:56:28,628 DEBUG [Listener at localhost/42557] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6a675bd6 to 127.0.0.1:64170 2023-02-09 09:56:28,628 DEBUG [Listener at localhost/42557] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:56:28,628 DEBUG [Listener at localhost/42557] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-09 09:56:28,628 DEBUG [Listener at localhost/42557] util.JVMClusterUtil(257): Found active master hash=335748510, stopped=false 2023-02-09 09:56:28,628 INFO [Listener at localhost/42557] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33303,1675936414925 2023-02-09 09:56:28,630 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 09:56:28,630 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 09:56:28,630 INFO [Listener at localhost/42557] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 09:56:28,630 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 09:56:28,630 DEBUG [Listener at localhost/42557] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x594f98ad to 127.0.0.1:64170 2023-02-09 09:56:28,630 DEBUG [Listener at localhost/42557] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:56:28,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 09:56:28,630 INFO [Listener at localhost/42557] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42719,1675936416061' ***** 2023-02-09 09:56:28,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 09:56:28,630 INFO [Listener at localhost/42557] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 09:56:28,631 INFO [RS:0;jenkins-hbase4:42719] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 09:56:28,631 INFO [RS:0;jenkins-hbase4:42719] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 09:56:28,631 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 09:56:28,631 INFO [RS:0;jenkins-hbase4:42719] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 09:56:28,631 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 954cfb87e30cb0d4ca9c9e7b1988704f 2023-02-09 09:56:28,632 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for bf230a91f2dd4768974bd854b33e048f 2023-02-09 09:56:28,632 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 1a102fc8ca7411175545f4844f76a455 2023-02-09 09:56:28,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 954cfb87e30cb0d4ca9c9e7b1988704f, disabling compactions & flushes 2023-02-09 09:56:28,632 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for e20e3543303de552ba42845d95488569 2023-02-09 09:56:28,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:56:28,632 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 7a50216fadfaa2886b6c84c6392b99cf 2023-02-09 09:56:28,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:56:28,632 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for bde72ffb8ef90a4019a13c516f644d0e 2023-02-09 09:56:28,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. after waiting 0 ms 2023-02-09 09:56:28,632 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 1b21e89c0550bd11bb849ead0d0c4c3d 2023-02-09 09:56:28,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 71fdc83e3110212c7e72a4e2bc3c2dea 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 47fec7fd4b7f8dc5df9e778c5af07aa9 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 18d34fcb29417f486e3561257b2790bf 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for d1ea6fc17b9a3bc112723fe71be3a078 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 2e714ed6a371d3ab3e132b9b4cdecb8f 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 651cd17c19c5ed7af78ba9deadfd9f44 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 1f71bdb54b6da843ebd313ec10a31be6 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 9e302a2c16508a1b912e987cb0942846 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 086d69d7e75a47257d3c163dd2bc28ef 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 60d93642fa0ceecf3be9bd92ffd4e009 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for e761366629ce85046adc8c312fb15374 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 335e253892ba08cb7cdfed22db4c2e83 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 927ca45314341fe03c1d443d120d31cf 2023-02-09 09:56:28,633 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:28,633 DEBUG [RS:0;jenkins-hbase4:42719] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4b22ac13 to 127.0.0.1:64170 2023-02-09 09:56:28,633 DEBUG [RS:0;jenkins-hbase4:42719] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:56:28,634 INFO [RS:0;jenkins-hbase4:42719] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 09:56:28,634 INFO [RS:0;jenkins-hbase4:42719] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 09:56:28,634 INFO [RS:0;jenkins-hbase4:42719] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 09:56:28,634 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-09 09:56:28,634 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-02-09 09:56:28,634 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 09:56:28,634 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 09:56:28,635 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 09:56:28,635 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 09:56:28,635 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 09:56:28,635 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-02-09 09:56:28,635 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1505): Waiting on 086d69d7e75a47257d3c163dd2bc28ef, 1588230740, 18d34fcb29417f486e3561257b2790bf, 1a102fc8ca7411175545f4844f76a455, 1b21e89c0550bd11bb849ead0d0c4c3d, 1f71bdb54b6da843ebd313ec10a31be6, 2e714ed6a371d3ab3e132b9b4cdecb8f, 335e253892ba08cb7cdfed22db4c2e83, 47fec7fd4b7f8dc5df9e778c5af07aa9, 60d93642fa0ceecf3be9bd92ffd4e009, 651cd17c19c5ed7af78ba9deadfd9f44, 71fdc83e3110212c7e72a4e2bc3c2dea, 7a50216fadfaa2886b6c84c6392b99cf, 927ca45314341fe03c1d443d120d31cf, 954cfb87e30cb0d4ca9c9e7b1988704f, 9e302a2c16508a1b912e987cb0942846, bde72ffb8ef90a4019a13c516f644d0e, bf230a91f2dd4768974bd854b33e048f, d1ea6fc17b9a3bc112723fe71be3a078, e20e3543303de552ba42845d95488569, e761366629ce85046adc8c312fb15374 2023-02-09 09:56:28,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/954cfb87e30cb0d4ca9c9e7b1988704f/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-09 09:56:28,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:56:28,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 954cfb87e30cb0d4ca9c9e7b1988704f: Waiting for close lock at 1675936588632Running coprocessor pre-close hooks at 1675936588632Disabling compacts and flushes for region at 1675936588632Disabling writes for close at 1675936588633 (+1 ms)Writing region close event to WAL at 1675936588633Running coprocessor post-close hooks at 1675936588637 (+4 ms)Closed at 1675936588638 (+1 ms) 2023-02-09 09:56:28,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1675936511349.954cfb87e30cb0d4ca9c9e7b1988704f. 2023-02-09 09:56:28,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bf230a91f2dd4768974bd854b33e048f, disabling compactions & flushes 2023-02-09 09:56:28,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:56:28,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:56:28,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. after waiting 0 ms 2023-02-09 09:56:28,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:56:28,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/bf230a91f2dd4768974bd854b33e048f/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-09 09:56:28,642 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:56:28,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bf230a91f2dd4768974bd854b33e048f: Waiting for close lock at 1675936588638Running coprocessor pre-close hooks at 1675936588638Disabling compacts and flushes for region at 1675936588638Disabling writes for close at 1675936588638Writing region close event to WAL at 1675936588639 (+1 ms)Running coprocessor post-close hooks at 1675936588642 (+3 ms)Closed at 1675936588642 2023-02-09 09:56:28,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675936488122.bf230a91f2dd4768974bd854b33e048f. 2023-02-09 09:56:28,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1a102fc8ca7411175545f4844f76a455, disabling compactions & flushes 2023-02-09 09:56:28,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:56:28,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:56:28,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. after waiting 0 ms 2023-02-09 09:56:28,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:56:28,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1a102fc8ca7411175545f4844f76a455/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-09 09:56:28,656 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:56:28,656 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1a102fc8ca7411175545f4844f76a455: Waiting for close lock at 1675936588643Running coprocessor pre-close hooks at 1675936588643Disabling compacts and flushes for region at 1675936588643Disabling writes for close at 1675936588643Writing region close event to WAL at 1675936588647 (+4 ms)Running coprocessor post-close hooks at 1675936588655 (+8 ms)Closed at 1675936588656 (+1 ms) 2023-02-09 09:56:28,656 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675936420123.1a102fc8ca7411175545f4844f76a455. 2023-02-09 09:56:28,656 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e20e3543303de552ba42845d95488569, disabling compactions & flushes 2023-02-09 09:56:28,656 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:56:28,656 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:56:28,656 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. after waiting 0 ms 2023-02-09 09:56:28,656 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:56:28,664 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e20e3543303de552ba42845d95488569/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 09:56:28,665 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:56:28,665 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e20e3543303de552ba42845d95488569: Waiting for close lock at 1675936588656Running coprocessor pre-close hooks at 1675936588656Disabling compacts and flushes for region at 1675936588656Disabling writes for close at 1675936588656Writing region close event to WAL at 1675936588661 (+5 ms)Running coprocessor post-close hooks at 1675936588664 (+3 ms)Closed at 1675936588665 (+1 ms) 2023-02-09 09:56:28,665 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1675936537898.e20e3543303de552ba42845d95488569. 2023-02-09 09:56:28,665 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7a50216fadfaa2886b6c84c6392b99cf, disabling compactions & flushes 2023-02-09 09:56:28,665 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:56:28,665 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:56:28,665 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. after waiting 0 ms 2023-02-09 09:56:28,665 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:56:28,668 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithMonthlyPolicy/7a50216fadfaa2886b6c84c6392b99cf/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-09 09:56:28,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:56:28,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7a50216fadfaa2886b6c84c6392b99cf: Waiting for close lock at 1675936588665Running coprocessor pre-close hooks at 1675936588665Disabling compacts and flushes for region at 1675936588665Disabling writes for close at 1675936588665Writing region close event to WAL at 1675936588666 (+1 ms)Running coprocessor post-close hooks at 1675936588668 (+2 ms)Closed at 1675936588669 (+1 ms) 2023-02-09 09:56:28,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1675936433873.7a50216fadfaa2886b6c84c6392b99cf. 2023-02-09 09:56:28,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bde72ffb8ef90a4019a13c516f644d0e, disabling compactions & flushes 2023-02-09 09:56:28,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:28,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:28,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. after waiting 0 ms 2023-02-09 09:56:28,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:28,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/bde72ffb8ef90a4019a13c516f644d0e/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 09:56:28,673 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:28,673 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bde72ffb8ef90a4019a13c516f644d0e: Waiting for close lock at 1675936588669Running coprocessor pre-close hooks at 1675936588669Disabling compacts and flushes for region at 1675936588669Disabling writes for close at 1675936588669Writing region close event to WAL at 1675936588670 (+1 ms)Running coprocessor post-close hooks at 1675936588672 (+2 ms)Closed at 1675936588673 (+1 ms) 2023-02-09 09:56:28,673 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1675936567540.bde72ffb8ef90a4019a13c516f644d0e. 2023-02-09 09:56:28,673 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1b21e89c0550bd11bb849ead0d0c4c3d, disabling compactions & flushes 2023-02-09 09:56:28,673 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:56:28,673 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:56:28,673 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. after waiting 0 ms 2023-02-09 09:56:28,673 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:56:28,675 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-02-09 09:56:28,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/1b21e89c0550bd11bb849ead0d0c4c3d/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-09 09:56:28,676 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:56:28,676 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1b21e89c0550bd11bb849ead0d0c4c3d: Waiting for close lock at 1675936588673Running coprocessor pre-close hooks at 1675936588673Disabling compacts and flushes for region at 1675936588673Disabling writes for close at 1675936588673Writing region close event to WAL at 1675936588673Running coprocessor post-close hooks at 1675936588676 (+3 ms)Closed at 1675936588676 2023-02-09 09:56:28,676 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1675936454542.1b21e89c0550bd11bb849ead0d0c4c3d. 2023-02-09 09:56:28,676 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 71fdc83e3110212c7e72a4e2bc3c2dea, disabling compactions & flushes 2023-02-09 09:56:28,676 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:56:28,676 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:56:28,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. after waiting 0 ms 2023-02-09 09:56:28,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:56:28,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/71fdc83e3110212c7e72a4e2bc3c2dea/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-09 09:56:28,683 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:56:28,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 71fdc83e3110212c7e72a4e2bc3c2dea: Waiting for close lock at 1675936588676Running coprocessor pre-close hooks at 1675936588676Disabling compacts and flushes for region at 1675936588676Disabling writes for close at 1675936588677 (+1 ms)Writing region close event to WAL at 1675936588677Running coprocessor post-close hooks at 1675936588682 (+5 ms)Closed at 1675936588682 2023-02-09 09:56:28,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675936472487.71fdc83e3110212c7e72a4e2bc3c2dea. 2023-02-09 09:56:28,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 47fec7fd4b7f8dc5df9e778c5af07aa9, disabling compactions & flushes 2023-02-09 09:56:28,683 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:56:28,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:56:28,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. after waiting 0 ms 2023-02-09 09:56:28,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:56:28,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/47fec7fd4b7f8dc5df9e778c5af07aa9/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-09 09:56:28,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:56:28,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 47fec7fd4b7f8dc5df9e778c5af07aa9: Waiting for close lock at 1675936588683Running coprocessor pre-close hooks at 1675936588683Disabling compacts and flushes for region at 1675936588683Disabling writes for close at 1675936588683Writing region close event to WAL at 1675936588684 (+1 ms)Running coprocessor post-close hooks at 1675936588686 (+2 ms)Closed at 1675936588687 (+1 ms) 2023-02-09 09:56:28,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1675936511349.47fec7fd4b7f8dc5df9e778c5af07aa9. 2023-02-09 09:56:28,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 18d34fcb29417f486e3561257b2790bf, disabling compactions & flushes 2023-02-09 09:56:28,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:28,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:28,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. after waiting 0 ms 2023-02-09 09:56:28,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:28,690 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/18d34fcb29417f486e3561257b2790bf/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 09:56:28,691 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:28,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 18d34fcb29417f486e3561257b2790bf: Waiting for close lock at 1675936588687Running coprocessor pre-close hooks at 1675936588687Disabling compacts and flushes for region at 1675936588687Disabling writes for close at 1675936588687Writing region close event to WAL at 1675936588688 (+1 ms)Running coprocessor post-close hooks at 1675936588691 (+3 ms)Closed at 1675936588691 2023-02-09 09:56:28,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1675936567540.18d34fcb29417f486e3561257b2790bf. 2023-02-09 09:56:28,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d1ea6fc17b9a3bc112723fe71be3a078, disabling compactions & flushes 2023-02-09 09:56:28,691 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:56:28,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:56:28,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. after waiting 0 ms 2023-02-09 09:56:28,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:56:28,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/d1ea6fc17b9a3bc112723fe71be3a078/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 09:56:28,695 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:56:28,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d1ea6fc17b9a3bc112723fe71be3a078: Waiting for close lock at 1675936588691Running coprocessor pre-close hooks at 1675936588691Disabling compacts and flushes for region at 1675936588691Disabling writes for close at 1675936588691Writing region close event to WAL at 1675936588692 (+1 ms)Running coprocessor post-close hooks at 1675936588695 (+3 ms)Closed at 1675936588695 2023-02-09 09:56:28,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1675936537898.d1ea6fc17b9a3bc112723fe71be3a078. 2023-02-09 09:56:28,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2e714ed6a371d3ab3e132b9b4cdecb8f, disabling compactions & flushes 2023-02-09 09:56:28,695 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:28,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:28,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. after waiting 0 ms 2023-02-09 09:56:28,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:28,698 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithMonthlyPolicy/2e714ed6a371d3ab3e132b9b4cdecb8f/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-09 09:56:28,699 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:28,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2e714ed6a371d3ab3e132b9b4cdecb8f: Waiting for close lock at 1675936588695Running coprocessor pre-close hooks at 1675936588695Disabling compacts and flushes for region at 1675936588695Disabling writes for close at 1675936588695Writing region close event to WAL at 1675936588696 (+1 ms)Running coprocessor post-close hooks at 1675936588698 (+2 ms)Closed at 1675936588699 (+1 ms) 2023-02-09 09:56:28,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1675936555913.2e714ed6a371d3ab3e132b9b4cdecb8f. 2023-02-09 09:56:28,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 651cd17c19c5ed7af78ba9deadfd9f44, disabling compactions & flushes 2023-02-09 09:56:28,699 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:56:28,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:56:28,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. after waiting 0 ms 2023-02-09 09:56:28,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:56:28,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionWithWeeklyPolicy/651cd17c19c5ed7af78ba9deadfd9f44/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 09:56:28,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:56:28,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 651cd17c19c5ed7af78ba9deadfd9f44: Waiting for close lock at 1675936588699Running coprocessor pre-close hooks at 1675936588699Disabling compacts and flushes for region at 1675936588699Disabling writes for close at 1675936588699Writing region close event to WAL at 1675936588700 (+1 ms)Running coprocessor post-close hooks at 1675936588702 (+2 ms)Closed at 1675936588703 (+1 ms) 2023-02-09 09:56:28,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1675936459163.651cd17c19c5ed7af78ba9deadfd9f44. 2023-02-09 09:56:28,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1f71bdb54b6da843ebd313ec10a31be6, disabling compactions & flushes 2023-02-09 09:56:28,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:56:28,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:56:28,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. after waiting 0 ms 2023-02-09 09:56:28,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:56:28,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMinorCompactionWithWeeklyPolicy/1f71bdb54b6da843ebd313ec10a31be6/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-02-09 09:56:28,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:56:28,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1f71bdb54b6da843ebd313ec10a31be6: Waiting for close lock at 1675936588703Running coprocessor pre-close hooks at 1675936588703Disabling compacts and flushes for region at 1675936588703Disabling writes for close at 1675936588703Writing region close event to WAL at 1675936588704 (+1 ms)Running coprocessor post-close hooks at 1675936588706 (+2 ms)Closed at 1675936588706 2023-02-09 09:56:28,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1675936445305.1f71bdb54b6da843ebd313ec10a31be6. 2023-02-09 09:56:28,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9e302a2c16508a1b912e987cb0942846, disabling compactions & flushes 2023-02-09 09:56:28,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:56:28,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:56:28,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. after waiting 0 ms 2023-02-09 09:56:28,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:56:28,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/9e302a2c16508a1b912e987cb0942846/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-09 09:56:28,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:56:28,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9e302a2c16508a1b912e987cb0942846: Waiting for close lock at 1675936588706Running coprocessor pre-close hooks at 1675936588706Disabling compacts and flushes for region at 1675936588706Disabling writes for close at 1675936588706Writing region close event to WAL at 1675936588707 (+1 ms)Running coprocessor post-close hooks at 1675936588709 (+2 ms)Closed at 1675936588709 2023-02-09 09:56:28,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1675936454542.9e302a2c16508a1b912e987cb0942846. 2023-02-09 09:56:28,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 086d69d7e75a47257d3c163dd2bc28ef, disabling compactions & flushes 2023-02-09 09:56:28,710 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:56:28,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:56:28,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. after waiting 0 ms 2023-02-09 09:56:28,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:56:28,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testScannerOnBulkLoadRefHFiles/086d69d7e75a47257d3c163dd2bc28ef/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-09 09:56:28,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:56:28,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 086d69d7e75a47257d3c163dd2bc28ef: Waiting for close lock at 1675936588709Running coprocessor pre-close hooks at 1675936588709Disabling compacts and flushes for region at 1675936588709Disabling writes for close at 1675936588710 (+1 ms)Writing region close event to WAL at 1675936588710Running coprocessor post-close hooks at 1675936588712 (+2 ms)Closed at 1675936588712 2023-02-09 09:56:28,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1675936454542.086d69d7e75a47257d3c163dd2bc28ef. 2023-02-09 09:56:28,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 60d93642fa0ceecf3be9bd92ffd4e009, disabling compactions & flushes 2023-02-09 09:56:28,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:56:28,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:56:28,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. after waiting 0 ms 2023-02-09 09:56:28,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:56:28,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testCompactionWithHFileLink/60d93642fa0ceecf3be9bd92ffd4e009/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-09 09:56:28,715 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:56:28,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 60d93642fa0ceecf3be9bd92ffd4e009: Waiting for close lock at 1675936588712Running coprocessor pre-close hooks at 1675936588712Disabling compacts and flushes for region at 1675936588712Disabling writes for close at 1675936588712Writing region close event to WAL at 1675936588713 (+1 ms)Running coprocessor post-close hooks at 1675936588715 (+2 ms)Closed at 1675936588715 2023-02-09 09:56:28,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1675936511349.60d93642fa0ceecf3be9bd92ffd4e009. 2023-02-09 09:56:28,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e761366629ce85046adc8c312fb15374, disabling compactions & flushes 2023-02-09 09:56:28,715 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:56:28,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:56:28,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. after waiting 0 ms 2023-02-09 09:56:28,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:56:28,716 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 09:56:28,718 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/ns/testMinorCompaction/e761366629ce85046adc8c312fb15374/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 09:56:28,718 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:56:28,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e761366629ce85046adc8c312fb15374: Waiting for close lock at 1675936588715Running coprocessor pre-close hooks at 1675936588715Disabling compacts and flushes for region at 1675936588715Disabling writes for close at 1675936588715Writing region close event to WAL at 1675936588716 (+1 ms)Running coprocessor post-close hooks at 1675936588718 (+2 ms)Closed at 1675936588718 2023-02-09 09:56:28,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1675936537898.e761366629ce85046adc8c312fb15374. 2023-02-09 09:56:28,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 335e253892ba08cb7cdfed22db4c2e83, disabling compactions & flushes 2023-02-09 09:56:28,719 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:28,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:28,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. after waiting 0 ms 2023-02-09 09:56:28,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:28,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/default/testMajorCompactionFromAdmin/335e253892ba08cb7cdfed22db4c2e83/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 09:56:28,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:28,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 335e253892ba08cb7cdfed22db4c2e83: Waiting for close lock at 1675936588719Running coprocessor pre-close hooks at 1675936588719Disabling compacts and flushes for region at 1675936588719Disabling writes for close at 1675936588719Writing region close event to WAL at 1675936588720 (+1 ms)Running coprocessor post-close hooks at 1675936588722 (+2 ms)Closed at 1675936588722 2023-02-09 09:56:28,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1675936567540.335e253892ba08cb7cdfed22db4c2e83. 2023-02-09 09:56:28,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 927ca45314341fe03c1d443d120d31cf, disabling compactions & flushes 2023-02-09 09:56:28,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:56:28,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:56:28,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. after waiting 0 ms 2023-02-09 09:56:28,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:56:28,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 927ca45314341fe03c1d443d120d31cf 1/1 column families, dataSize=109 B heapSize=592 B 2023-02-09 09:56:28,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf/.tmp/info/cfe526db91834a68a2e9e3f243c1c080 2023-02-09 09:56:28,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf/.tmp/info/cfe526db91834a68a2e9e3f243c1c080 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf/info/cfe526db91834a68a2e9e3f243c1c080 2023-02-09 09:56:28,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf/info/cfe526db91834a68a2e9e3f243c1c080, entries=3, sequenceid=7, filesize=4.8 K 2023-02-09 09:56:28,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 927ca45314341fe03c1d443d120d31cf in 15ms, sequenceid=7, compaction requested=false 2023-02-09 09:56:28,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/namespace/927ca45314341fe03c1d443d120d31cf/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-02-09 09:56:28,742 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:56:28,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 927ca45314341fe03c1d443d120d31cf: Waiting for close lock at 1675936588722Running coprocessor pre-close hooks at 1675936588722Disabling compacts and flushes for region at 1675936588722Disabling writes for close at 1675936588722Obtaining lock to block concurrent updates at 1675936588722Preparing flush snapshotting stores in 927ca45314341fe03c1d443d120d31cf at 1675936588722Finished memstore snapshotting hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf., syncing WAL and waiting on mvcc, flushsize=dataSize=109, getHeapSize=576, getOffHeapSize=0, getCellsCount=3 at 1675936588722Flushing stores of hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. at 1675936588723 (+1 ms)Flushing 927ca45314341fe03c1d443d120d31cf/info: creating writer at 1675936588723Flushing 927ca45314341fe03c1d443d120d31cf/info: appending metadata at 1675936588724 (+1 ms)Flushing 927ca45314341fe03c1d443d120d31cf/info: closing flushed file at 1675936588724Flushing 927ca45314341fe03c1d443d120d31cf/info: reopening flushed file at 1675936588732 (+8 ms)Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 927ca45314341fe03c1d443d120d31cf in 15ms, sequenceid=7, compaction requested=false at 1675936588737 (+5 ms)Writing region close event to WAL at 1675936588739 (+2 ms)Running coprocessor post-close hooks at 1675936588742 (+3 ms)Closed at 1675936588742 2023-02-09 09:56:28,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675936418966.927ca45314341fe03c1d443d120d31cf. 2023-02-09 09:56:28,835 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 09:56:29,036 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 09:56:29,056 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/.tmp/info/890626f8467348d28999b8a92583d22f 2023-02-09 09:56:29,059 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 890626f8467348d28999b8a92583d22f 2023-02-09 09:56:29,236 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 09:56:29,358 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 09:56:29,436 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 09:56:29,465 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/.tmp/rep_barrier/9c2cc73dc0f546a1b98d6999e5064d90 2023-02-09 09:56:29,468 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9c2cc73dc0f546a1b98d6999e5064d90 2023-02-09 09:56:29,630 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 09:56:29,630 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 09:56:29,637 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-09 09:56:29,637 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-09 09:56:29,637 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 09:56:29,837 DEBUG [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 09:56:29,874 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/.tmp/table/58a1a79ec3ca46f19e158b4b268e5901 2023-02-09 09:56:29,877 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58a1a79ec3ca46f19e158b4b268e5901 2023-02-09 09:56:29,878 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/.tmp/info/890626f8467348d28999b8a92583d22f as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/info/890626f8467348d28999b8a92583d22f 2023-02-09 09:56:29,881 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 890626f8467348d28999b8a92583d22f 2023-02-09 09:56:29,881 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/info/890626f8467348d28999b8a92583d22f, entries=226, sequenceid=130, filesize=37.9 K 2023-02-09 09:56:29,881 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/.tmp/rep_barrier/9c2cc73dc0f546a1b98d6999e5064d90 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/rep_barrier/9c2cc73dc0f546a1b98d6999e5064d90 2023-02-09 09:56:29,884 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9c2cc73dc0f546a1b98d6999e5064d90 2023-02-09 09:56:29,884 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/rep_barrier/9c2cc73dc0f546a1b98d6999e5064d90, entries=1, sequenceid=130, filesize=5.0 K 2023-02-09 09:56:29,885 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/.tmp/table/58a1a79ec3ca46f19e158b4b268e5901 as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/table/58a1a79ec3ca46f19e158b4b268e5901 2023-02-09 09:56:29,888 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58a1a79ec3ca46f19e158b4b268e5901 2023-02-09 09:56:29,888 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/table/58a1a79ec3ca46f19e158b4b268e5901, entries=27, sequenceid=130, filesize=6.6 K 2023-02-09 09:56:29,890 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1255ms, sequenceid=130, compaction requested=false 2023-02-09 09:56:29,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-02-09 09:56:29,893 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 09:56:29,894 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 09:56:29,894 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675936588634Running coprocessor pre-close hooks at 1675936588634Disabling compacts and flushes for region at 1675936588634Disabling writes for close at 1675936588635 (+1 ms)Obtaining lock to block concurrent updates at 1675936588635Preparing flush snapshotting stores in 1588230740 at 1675936588635Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=51772, getHeapSize=80064, getOffHeapSize=0, getCellsCount=365 at 1675936588635Flushing stores of hbase:meta,,1.1588230740 at 1675936588635Flushing 1588230740/info: creating writer at 1675936588635Flushing 1588230740/info: appending metadata at 1675936588639 (+4 ms)Flushing 1588230740/info: closing flushed file at 1675936588639Flushing 1588230740/rep_barrier: creating writer at 1675936589059 (+420 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675936589061 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675936589061Flushing 1588230740/table: creating writer at 1675936589468 (+407 ms)Flushing 1588230740/table: appending metadata at 1675936589470 (+2 ms)Flushing 1588230740/table: closing flushed file at 1675936589470Flushing 1588230740/info: reopening flushed file at 1675936589878 (+408 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675936589882 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1675936589885 (+3 ms)Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1255ms, sequenceid=130, compaction requested=false at 1675936589890 (+5 ms)Writing region close event to WAL at 1675936589891 (+1 ms)Running coprocessor post-close hooks at 1675936589893 (+2 ms)Closed at 1675936589894 (+1 ms) 2023-02-09 09:56:29,894 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-09 09:56:30,037 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42719,1675936416061; all regions closed. 2023-02-09 09:56:30,045 DEBUG [RS:0;jenkins-hbase4:42719] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/oldWALs 2023-02-09 09:56:30,045 INFO [RS:0;jenkins-hbase4:42719] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42719%2C1675936416061.meta:.meta(num 1675936418716) 2023-02-09 09:56:30,048 DEBUG [RS:0;jenkins-hbase4:42719] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/oldWALs 2023-02-09 09:56:30,048 INFO [RS:0;jenkins-hbase4:42719] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42719%2C1675936416061:(num 1675936417779) 2023-02-09 09:56:30,048 DEBUG [RS:0;jenkins-hbase4:42719] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:56:30,048 INFO [RS:0;jenkins-hbase4:42719] regionserver.LeaseManager(133): Closed leases 2023-02-09 09:56:30,049 INFO [RS:0;jenkins-hbase4:42719] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-09 09:56:30,049 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 09:56:30,050 INFO [RS:0;jenkins-hbase4:42719] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42719 2023-02-09 09:56:30,058 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42719,1675936416061 2023-02-09 09:56:30,058 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 09:56:30,058 ERROR [Listener at localhost/42557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@a433565 rejected from java.util.concurrent.ThreadPoolExecutor@e7ca0f5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 09:56:30,058 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 09:56:30,058 ERROR [Listener at localhost/42557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@69841d57 rejected from java.util.concurrent.ThreadPoolExecutor@e7ca0f5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 09:56:30,059 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42719,1675936416061] 2023-02-09 09:56:30,059 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42719,1675936416061; numProcessing=1 2023-02-09 09:56:30,060 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42719,1675936416061 already deleted, retry=false 2023-02-09 09:56:30,060 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42719,1675936416061 expired; onlineServers=0 2023-02-09 09:56:30,060 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33303,1675936414925' ***** 2023-02-09 09:56:30,060 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-09 09:56:30,061 DEBUG [M:0;jenkins-hbase4:33303] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@553e17a5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 09:56:30,061 INFO [M:0;jenkins-hbase4:33303] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33303,1675936414925 2023-02-09 09:56:30,061 INFO [M:0;jenkins-hbase4:33303] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33303,1675936414925; all regions closed. 2023-02-09 09:56:30,061 DEBUG [M:0;jenkins-hbase4:33303] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:56:30,061 DEBUG [M:0;jenkins-hbase4:33303] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-09 09:56:30,061 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-09 09:56:30,061 DEBUG [M:0;jenkins-hbase4:33303] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-09 09:56:30,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675936417457] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675936417457,5,FailOnTimeoutGroup] 2023-02-09 09:56:30,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675936417457] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675936417457,5,FailOnTimeoutGroup] 2023-02-09 09:56:30,061 INFO [M:0;jenkins-hbase4:33303] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-09 09:56:30,061 INFO [M:0;jenkins-hbase4:33303] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-09 09:56:30,062 INFO [M:0;jenkins-hbase4:33303] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-09 09:56:30,062 DEBUG [M:0;jenkins-hbase4:33303] master.HMaster(1502): Stopping service threads 2023-02-09 09:56:30,062 INFO [M:0;jenkins-hbase4:33303] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-09 09:56:30,062 ERROR [M:0;jenkins-hbase4:33303] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-02-09 09:56:30,063 INFO [M:0;jenkins-hbase4:33303] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 09:56:30,063 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-09 09:56:30,066 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 09:56:30,066 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 09:56:30,066 DEBUG [M:0;jenkins-hbase4:33303] zookeeper.ZKUtil(398): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-09 09:56:30,066 WARN [M:0;jenkins-hbase4:33303] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-09 09:56:30,066 INFO [M:0;jenkins-hbase4:33303] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 09:56:30,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 09:56:30,066 INFO [M:0;jenkins-hbase4:33303] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-09 09:56:30,066 DEBUG [M:0;jenkins-hbase4:33303] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 09:56:30,066 INFO [M:0;jenkins-hbase4:33303] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 09:56:30,066 DEBUG [M:0;jenkins-hbase4:33303] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 09:56:30,066 DEBUG [M:0;jenkins-hbase4:33303] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 09:56:30,066 DEBUG [M:0;jenkins-hbase4:33303] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 09:56:30,066 INFO [M:0;jenkins-hbase4:33303] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.79 KB heapSize=598.60 KB 2023-02-09 09:56:30,073 INFO [M:0;jenkins-hbase4:33303] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.79 KB at sequenceid=1006 (bloomFilter=true), to=hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/c135a53bd13446569b7ab1f5ba1a020e 2023-02-09 09:56:30,076 INFO [M:0;jenkins-hbase4:33303] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c135a53bd13446569b7ab1f5ba1a020e 2023-02-09 09:56:30,077 DEBUG [M:0;jenkins-hbase4:33303] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/c135a53bd13446569b7ab1f5ba1a020e as hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/c135a53bd13446569b7ab1f5ba1a020e 2023-02-09 09:56:30,079 INFO [M:0;jenkins-hbase4:33303] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c135a53bd13446569b7ab1f5ba1a020e 2023-02-09 09:56:30,079 INFO [M:0;jenkins-hbase4:33303] regionserver.HStore(1080): Added hdfs://localhost:37539/user/jenkins/test-data/2e132d9e-69bf-419f-480f-22e32346633b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/c135a53bd13446569b7ab1f5ba1a020e, entries=123, sequenceid=1006, filesize=33.3 K 2023-02-09 09:56:30,080 INFO [M:0;jenkins-hbase4:33303] regionserver.HRegion(2947): Finished flush of dataSize ~512.79 KB/525092, heapSize ~598.59 KB/612952, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 14ms, sequenceid=1006, compaction requested=false 2023-02-09 09:56:30,081 INFO [M:0;jenkins-hbase4:33303] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 09:56:30,081 DEBUG [M:0;jenkins-hbase4:33303] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675936590066Disabling compacts and flushes for region at 1675936590066Disabling writes for close at 1675936590066Obtaining lock to block concurrent updates at 1675936590066Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675936590066Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=525092, getHeapSize=612952, getOffHeapSize=0, getCellsCount=1161 at 1675936590067 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675936590067Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675936590067Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675936590070 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675936590070Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675936590077 (+7 ms)Finished flush of dataSize ~512.79 KB/525092, heapSize ~598.59 KB/612952, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 14ms, sequenceid=1006, compaction requested=false at 1675936590080 (+3 ms)Writing region close event to WAL at 1675936590081 (+1 ms)Closed at 1675936590081 2023-02-09 09:56:30,083 INFO [M:0;jenkins-hbase4:33303] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-09 09:56:30,083 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 09:56:30,084 INFO [M:0;jenkins-hbase4:33303] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33303 2023-02-09 09:56:30,086 DEBUG [M:0;jenkins-hbase4:33303] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33303,1675936414925 already deleted, retry=false 2023-02-09 09:56:30,159 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 09:56:30,159 INFO [RS:0;jenkins-hbase4:42719] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42719,1675936416061; zookeeper connection closed. 2023-02-09 09:56:30,159 ERROR [Listener at localhost/42557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@11ef89bd rejected from java.util.concurrent.ThreadPoolExecutor@e7ca0f5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 09:56:30,159 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3eea2550] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3eea2550 2023-02-09 09:56:30,160 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): regionserver:42719-0x100388023520001, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 09:56:30,160 ERROR [Listener at localhost/42557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@62f60e89 rejected from java.util.concurrent.ThreadPoolExecutor@e7ca0f5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 09:56:30,160 INFO [Listener at localhost/42557] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-02-09 09:56:30,259 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 09:56:30,259 INFO [M:0;jenkins-hbase4:33303] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33303,1675936414925; zookeeper connection closed. 2023-02-09 09:56:30,259 ERROR [Listener at localhost/42557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7c6e1aaf rejected from java.util.concurrent.ThreadPoolExecutor@2a20b362[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 09:56:30,259 DEBUG [Listener at localhost/42557-EventThread] zookeeper.ZKWatcher(600): master:33303-0x100388023520000, quorum=127.0.0.1:64170, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 09:56:30,260 ERROR [Listener at localhost/42557-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@12877b3f rejected from java.util.concurrent.ThreadPoolExecutor@2a20b362[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 09:56:30,261 WARN [Listener at localhost/42557] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 09:56:30,264 INFO [Listener at localhost/42557] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 09:56:30,371 WARN [BP-294638041-172.31.14.131-1675936411864 heartbeating to localhost/127.0.0.1:37539] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 09:56:30,371 WARN [BP-294638041-172.31.14.131-1675936411864 heartbeating to localhost/127.0.0.1:37539] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-294638041-172.31.14.131-1675936411864 (Datanode Uuid 41a2457e-4972-4171-9111-56b0322eb7a3) service to localhost/127.0.0.1:37539 2023-02-09 09:56:30,374 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/cluster_355d5ed8-3bf0-2bc4-8778-241b2c013167/dfs/data/data1/current/BP-294638041-172.31.14.131-1675936411864] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 09:56:30,374 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0f3a44fd-7e27-8c12-6484-fd483fe87390/cluster_355d5ed8-3bf0-2bc4-8778-241b2c013167/dfs/data/data2/current/BP-294638041-172.31.14.131-1675936411864] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 09:56:30,404 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 09:56:30,404 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x096775fe to 127.0.0.1:64170 2023-02-09 09:56:30,404 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 09:56:30,420 INFO [Listener at localhost/42557] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 09:56:30,549 INFO [Listener at localhost/42557] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-09 09:56:30,628 INFO [Listener at localhost/42557] hbase.HBaseTestingUtility(1293): Minicluster is down